Continual Learning

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

33 leaderboard tables shown for this task, 33 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted. 10 shown of 33 until expanded.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

35 datasets whose archive record lists this task, ordered by the archive's paper count. 30 shown of 35 until expanded.

Subtasks archive 2025-07-28

5 subtasks in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 1,142 papers with code (2,644 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Overcoming catastrophic forgetting in neural networks
  2 Dec 2016 29 repositories listed Syntology ran 14 of 22 samples · 8 unverified · 4 pointer-only (licence)
  The ability to learn tasks in a sequential fashion is crucial to the development of artificial intelligence.
• Learning without Forgetting
  29 Jun 2016 12 repositories listed Syntology ran 9 of 15 samples · 6 unverified · 1 pointer-only (licence)
  We propose our Learning without Forgetting method, which uses only new task data to train the network while preserving the original capabilities.
• Progressive Neural Networks
  15 Jun 2016 12 repositories listed Syntology ran 0 of 3 samples · 3 unverified · 3 pointer-only (licence)
  Learning to solve complex sequences of tasks--while both leveraging transfer and avoiding catastrophic forgetting--remains a key obstacle to achieving human-level intelligence.
• Continual learning with hypernetworks
  3 Jun 2019 9 repositories listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  Artificial neural networks suffer from catastrophic forgetting when they are sequentially trained on multiple tasks.
• Three scenarios for continual learning
  15 Apr 2019 8 repositories listed Syntology ran 0 of 2 samples · 2 unverified
  Standard artificial neural networks suffer from the well-known issue of catastrophic forgetting, making continual or lifelong learning difficult for machine learning.
• Variational Continual Learning
  29 Oct 2017 8 repositories listed Syntology ran 3 of 4 samples · 1 unverified · 3 pointer-only (licence)
  This paper develops variational continual learning (VCL), a simple but general framework for continual learning that fuses online variational inference (VI) and recent advances in Monte Carlo VI for neural networks.
• Gradient Episodic Memory for Continual Learning
  26 Jun 2017 7 repositories listed
  One major obstacle towards AI is the poor ability of models to solve new problems quicker, and without forgetting previously acquired knowledge.
• Meta-Learning Representations for Continual Learning
  29 May 2019 6 repositories listed
  We show that it is possible to learn naturally sparse representations that are more effective for online updating.
• On Tiny Episodic Memories in Continual Learning
  27 Feb 2019 6 repositories listed
  But for a successful knowledge transfer, the learner needs to remember how to perform previous tasks.
• Continual Learning Through Synaptic Intelligence
  13 Mar 2017 6 repositories listed Syntology ran 0 of 7 samples · 7 unverified
  While deep learning has led to remarkable advances across diverse applications, it struggles in domains where the data distribution changes over the course of learning.
• BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage
  5 Aug 2022 5 repositories listed
  We present BlenderBot 3, a 175B parameter dialogue model capable of open-domain conversation with access to the internet and a long-term memory, and having been trained on a large number of user defined tasks.
• Learning to Prompt for Continual Learning
  16 Dec 2021 5 repositories listed Syntology ran 10 of 11 samples · 1 unverified · 10 pointer-only (licence)
  The mainstream paradigm behind continual learning has been to adapt the model parameters to non-stationary data distributions, where catastrophic forgetting is the central challenge.
• Dataset Condensation with Gradient Matching
  10 Jun 2020 5 repositories listed
  As the state-of-the-art machine learning methods in many fields rely on larger datasets, storing datasets and training models on them become significantly more expensive.
• Learning to Continually Learn
  21 Feb 2020 5 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 3 pointer-only (licence)
  Continual lifelong learning requires an agent or model to learn many sequentially ordered tasks, building on previous knowledge without catastrophically forgetting it.
• Rehearsal-Free Continual Learning over Small Non-I.I.D. Batches
  8 Jul 2019 5 repositories listed
  Ideally, continual learning should be triggered by the availability of short videos of single objects and performed on-line on on-board hardware with fine-grained updates.
• Gradient based sample selection for online continual learning
  20 Mar 2019 5 repositories listed
  To prevent forgetting, a replay buffer is usually employed to store the previous data for the purpose of rehearsal.
• Generative replay with feedback connections as a general strategy for continual learning
  27 Sep 2018 5 repositories listed
  A major obstacle to developing artificial intelligence applications capable of true lifelong learning is that artificial neural networks quickly or catastrophically forget previously learned tasks when trained on a new…
• Continual Learning with Deep Generative Replay
  24 May 2017 5 repositories listed
  Attempts to train a comprehensive artificial intelligence capable of solving multiple tasks have been impeded by a chronic problem called catastrophic forgetting.
• Dataset Condensation with Distribution Matching
  8 Oct 2021 4 repositories listed
  Computational cost of training state-of-the-art deep models in many learning problems is rapidly increasing due to more sophisticated models and larger datasets.
• Avalanche: an End-to-End Library for Continual Learning
  1 Apr 2021 4 repositories listed Syntology ran 3 of 14 samples · 11 unverified
  Learning continually from non-stationary data streams is a long-standing goal and a challenging problem in machine learning.
• Learning to Continuously Optimize Wireless Resource In Episodically Dynamic Environment
  16 Nov 2020 4 repositories listed
  We propose to build the notion of continual learning (CL) into the modeling process of learning wireless systems, so that the learning model can incrementally adapt to the new episodes, {\it without forgetting}…
• Understanding the Role of Training Regimes in Continual Learning
  12 Jun 2020 4 repositories listed Syntology ran 2 of 20 samples · 18 unverified
  However, there has been limited prior work extensively analyzing the impact that different training regimes -- learning rate, batch size, regularization method-- can have on forgetting.
• Training Binary Neural Networks using the Bayesian Learning Rule
  25 Feb 2020 4 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 3 pointer-only (licence)
  Our work provides a principled approach for training binary neural networks which justifies and extends existing approaches.
• Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learning
  1 Jul 2019 4 repositories listed Syntology ran 1 of 12 samples · 11 unverified · 2 pointer-only (licence)
  The Radial BNN is motivated by avoiding a sampling problem in 'mean-field' variational inference (MFVI) caused by the so-called 'soap-bubble' pathology of multivariate Gaussians.
• PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning
  15 Nov 2017 4 repositories listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  This paper presents a method for adding multiple tasks to a single deep neural network while avoiding catastrophic forgetting.
• FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learning
  25 Sep 2023 3 repositories listed Syntology ran 11 of 21 samples · 10 unverified
  However, when learning from non-stationary data, we observe that the Euclidean metric is suboptimal and that feature distributions are heterogeneous.
• CaT: Balanced Continual Graph Learning with Graph Condensation
  18 Sep 2023 3 repositories listed Syntology ran 0 of 1 samples · 1 unverified · 1 pointer-only (licence)
  Recent replay-based methods intend to solve this problem by updating the model using both (1) the entire new-coming data and (2) a sampling-based memory bank that stores replayed graphs to approximate the distribution…
• Class-Incremental Learning: A Survey
  7 Feb 2023 3 repositories listed
  Deep models, e.
• Latent Spectral Regularization for Continual Learning
  9 Jan 2023 3 repositories listed
  While biological intelligence grows organically as new knowledge is gathered throughout life, Artificial Neural Networks forget catastrophically whenever they face a changing training data distribution.
• Continual Training of Language Models for Few-Shot Learning
  11 Oct 2022 3 repositories listed
  Recent work on applying large language models (LMs) achieves impressive performance in many NLP applications.

Syntology lines on 16 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.