Papers › Learning Bayesian Sparse Networks with Full Experience Replay for Continual Learning
Learning Bayesian Sparse Networks with Full Experience Replay for Continual Learning
Dong Gong, Qingsen Yan, Yuhang Liu, Anton Van Den Hengel, Javen Qinfeng Shi
Continual Learning (CL) methods aim to enable machine learning models to learn new tasks without catastrophic forgetting of those that have been previously mastered. Existing CL approaches often keep a buffer of previously-seen samples, perform knowledge distillation, or use regularization techniques towards this goal. Despite their performance, they still suffer from interference across tasks which leads to catastrophic forgetting. To ameliorate this problem, we propose to only activate and select sparse neurons for learning current and past tasks at any stage. More parameters space and model capacity can thus be reserved for the future tasks. This minimizes the interference between parameters for different tasks. To do so, we propose a Sparse neural Network for Continual Learning (SNCL), which employs variational Bayesian sparsity priors on the activations of the neurons in all layers. Full Experience Replay (FER) provides effective supervision in learning the sparse activations of the neurons in different layers. A loss-aware reservoir-sampling strategy is developed to maintain the memory buffer. The proposed method is agnostic as to the network structures and the task boundaries. Experiments on different datasets show that our approach achieves state-of-the-art performance for mitigating forgetting.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Continual Learning | Tiny-ImageNet (10tasks) | SNCL | Average Accuracy | 52.85 | #5 of 9 | Archive leaderboard | report |
| Continual Learning | Tiny-ImageNet (10tasks) | DER [buzzega2020dark] | Average Accuracy | 51.78 | #6 of 9 | Archive leaderboard | report |
| Continual Learning | Tiny-ImageNet (10tasks) | ER[riemer2018learning] | Average Accuracy | 48.64 | #7 of 9 | Archive leaderboard | report |
| Continual Learning | Tiny-ImageNet (10tasks) | iCaRL [rebuffi2017icarl] | Average Accuracy | 31.55 | #8 of 9 | Archive leaderboard | report |
| Continual Learning | Tiny-ImageNet (10tasks) | A-GEM [chaudhry2018efficient] | Average Accuracy | 25.33 | #9 of 9 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections