Papers › Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning
Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning
Zixuan Ke, Bing Liu, Nianzu Ma, Hu Xu, Lei Shu
Continual learning (CL) learns a sequence of tasks incrementally with the goal of achieving two main objectives: overcoming catastrophic forgetting (CF) and encouraging knowledge transfer (KT) across tasks. However, most existing techniques focus only on overcoming CF and have no mechanism to encourage KT, and thus do not do well in KT. Although several papers have tried to deal with both CF and KT, our experiments show that they suffer from serious CF when the tasks do not have much shared knowledge. Another observation is that most current CL methods do not use pre-trained models, but it has been shown that such models can significantly improve the end task performance. For example, in natural language processing, fine-tuning a BERT-like pre-trained language model is one of the most effective approaches. However, for CL, this approach suffers from serious CF. An interesting question is how to make the best use of pre-trained models for CL. This paper proposes a novel model called CTR to solve these problems. Our experimental results demonstrate the effectiveness of CTR
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Datasets
Introduced by this paper, per the archive.
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Continual Learning | 20Newsgroup (10 tasks) | CTR | F1 - macro | 0.9523 | #1 of 6 | Archive leaderboard | report |
| Continual Learning | ASC (19 tasks) | Multi-task Learning (MTL; Upper Bound) | F1 - macro | 0.8811 | #1 of 15 | Archive leaderboard | report |
| Continual Learning | ASC (19 tasks) | CTR | F1 - macro | 0.8362 | #2 of 15 | Archive leaderboard | report |
| Continual Learning | ASC (19 tasks) | Independent Learning (ONE) | F1 - macro | 0.7807 | #8 of 15 | Archive leaderboard | report |
| Continual Learning | ASC (19 tasks) | Naive Continual Learning (NCL) | F1 - macro | 0.7664 | #10 of 15 | Archive leaderboard | report |
| Continual Learning | DSC (10 tasks) | CTR | F1 - macro | 0.8875 | #1 of 6 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections