Papers › TAG: Task-based Accumulated Gradients for Lifelong learning
TAG: Task-based Accumulated Gradients for Lifelong learning
Pranshu Malviya, Balaraman Ravindran, Sarath Chandar
When an agent encounters a continual stream of new tasks in the lifelong learning setting, it leverages the knowledge it gained from the earlier tasks to help learn the new tasks better. In such a scenario, identifying an efficient knowledge representation becomes a challenging problem. Most research works propose to either store a subset of examples from the past tasks in a replay buffer, dedicate a separate set of parameters to each task or penalize excessive updates over parameters by introducing a regularization term. While existing methods employ the general task-agnostic stochastic gradient descent update rule, we propose a task-aware optimizer that adapts the learning rate based on the relatedness among tasks. We utilize the directions taken by the parameters during the updates by accumulating the gradients specific to each task. These task-based accumulated gradients act as a knowledge base that is maintained and updated throughout the stream. We empirically show that our proposed adaptive learning rate not only accounts for catastrophic forgetting but also allows positive backward transfer. We also show that our method performs better than several state-of-the-art methods in lifelong learning on complex datasets with a large number of tasks.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Continual Learning | 5-dataset - 1 epoch | TAG-RMSProp | Accuracy | 62.59 | #1 of 1 | Archive leaderboard | report |
| Continual Learning | CUB-200-2011 (20 tasks) - 1 epoch | TAG-RMSProp | Accuracy | 61.58 | #1 of 1 | Archive leaderboard | report |
| Continual Learning | Cifar100 (20 tasks) - 1 epoch | TAG-RMSProp | Average Accuracy | 62.79 | #1 of 1 | Archive leaderboard | report |
| Continual Learning | mini-Imagenet (20 tasks) - 1 epoch | TAG-RMSProp | Accuracy | 57.2 | #1 of 1 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections