Papers › Overcoming catastrophic forgetting in neural networks
Overcoming catastrophic forgetting in neural networks
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, Raia Hadsell
The ability to learn tasks in a sequential fashion is crucial to the development of artificial intelligence. Neural networks are not, in general, capable of this and it has been widely thought that catastrophic forgetting is an inevitable feature of connectionist models. We show that it is possible to overcome this limitation and train networks that can maintain expertise on tasks which they have not experienced for a long time. Our approach remembers old tasks by selectively slowing down learning on the weights important for those tasks. We demonstrate our approach is scalable and effective by solving a set of classification tasks based on the MNIST hand written digit dataset and by learning several Atari 2600 games sequentially.
In Syntology View this paper on Syntology: its repositories, every harvested function with whether it ran, its licence and the call to fetch it.
Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
For agents, Syntology's MCP tool lists every function and class Syntology harvested from this paper and whether it ran (how to connect): get_harvested_code_for_paper(arxiv_id="1612.00796")
Code
Syntology Ran 14 of 22 code samples harvested from 12 repositories linked to this paper; 8 have no recorded run. Of those that ran: 2 ran · honoured contract; 4 ran · our draft was wrong; 8 ran with no contract checked.
By repository: community (archive-listed): 22 samples from 12 repositories, 14 ran. The run record, sample by sample. “Ran” means executed on a synthesized input, not that the code is correct or reproduces the paper.
29 repositories listed; official and paper-mentioned ones first.
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
22 samples harvested; 14 ran; 2 honoured the contract we drafted; 8 have no recorded run. Read from Syntology's graph 2026-09-25; that is when this build read the record, not when the samples ran.
Licence: 4 of the 22 samples are pointer only, meaning Syntology does not serve that copy's text. This page shows no code text for any sample; each one links to its file in the repository.
Harvested from 12 repositories linked to this paper, official or community; each sample names its own and says which. “Ran” means the sample executed on a synthesized input. It does not mean the output is correct, and nothing here reproduces the paper's results. “Honoured” and “violated” refer to a contract Syntology drafted from the code itself; “our draft was wrong” and “fixture could not drive it” are failures of Syntology's instrument, not of the code.
Each sample ends with its code_sha256, Syntology's identity for that exact code. An agent fetches the stored sample with Syntology's MCP tool get_code(code_sha256="…") (how to connect); click an identity to copy that call.
Repository labels, per sample. official repository: The archive marks this repository official for the paper. named in the paper: The archive records that the paper mentions this repository; it is not marked official. community (archive-listed): In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper. found in paper text by Syntology: Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted. community: Not in the archive's code links for this paper; a community repository Syntology harvested. Samples from a repository marked official are listed first. Licence labels name the repository's licence as recorded at harvest. “Pointer only” means Syntology does not serve that copy's text, for one of four reasons: no licence file was found; the licence was not identified; the licence is recorded as permissive but that copy's record is not marked cleared; or the licence is outside the permissive list Syntology serves text under (MIT, Apache-2.0, BSD and similar). Some licences outside that list permit redistribution, such as WTFPL, and GPL-3.0 under its conditions; they are simply not on the list. Hover a licence label for the reason. File links open the file on GitHub at the default branch, which may have changed since the harvest.
3ca3b2ad5cf08e7a · report
a1465fe6cdcb57d7 · report
6aad701239ed7577 · report
39b8751422fdac64 · report
1d10e03aa48307d3 · report
b26c848baf0f673d · report
a47c1a346ec772ca · report
c1ecdf9c6eed882e · report
b7d9d94554e93b1b · report
a1703c053471f8d6 · report
e25911109ba29783 · report
4473672cd588ecd4 · report
426df1649f33777b · report
aff2576823d89d61 · report
c1596b67340a6f97 · report
118dad0481c8d9e3 · report
5ebc601b0d506a2c · report
9d058983f0e0d999 · report
cbe2eeebdcb6a48a · report
7cfca855faa8e8dc · report
5e7be12170048fbb · report
ca1097ecd04eb4a3 · report
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Continual Learning | 20Newsgroup (10 tasks) | EWC | F1 - macro | 0.9180 | #5 of 6 | Archive leaderboard | report |
| Continual Learning | ASC (19 tasks) | EWC | F1 - macro | 0.7452 | #13 of 15 | Archive leaderboard | report |
| Continual Learning | ASC (19 tasks) | L2 | F1 - macro | 0.5243 | #15 of 15 | Archive leaderboard | report |
| Continual Learning | DSC (10 tasks) | EWC | F1 - macro | 0.6576 | #6 of 6 | Archive leaderboard | report |
| Continual Learning | F-CelebA (10 tasks) | EWC | Acc | 0.6545 | #3 of 7 | Archive leaderboard | report |
| class-incremental learning | cifar100 | EWC | 10-stage average accuracy | 50.53 | #1 of 1 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Introduced by this paper: EWC
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections