{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/overcoming-long-term-catastrophic-forgetting","title":"Overcoming Long-term Catastrophic Forgetting through Adversarial Neural Pruning and Synaptic Consolidation","arxiv_id":"1912.09091","date":"2019-12-19","proceeding":null,"authors":["Jian Peng","Bo Tang","Hao Jiang","Zhuo Li","Yinjie Lei","Tao Lin","Haifeng Li"],"abstract":"Artificial neural networks face the well-known problem of catastrophic forgetting. What's worse, the degradation of previously learned skills becomes more severe as the task sequence increases, known as the long-term catastrophic forgetting. It is due to two facts: first, as the model learns more tasks, the intersection of the low-error parameter subspace satisfying for these tasks becomes smaller or even does not exist; second, when the model learns a new task, the cumulative error keeps increasing as the model tries to protect the parameter configuration of previous tasks from interference. Inspired by the memory consolidation mechanism in mammalian brains with synaptic plasticity, we propose a confrontation mechanism in which Adversarial Neural Pruning and synaptic Consolidation (ANPyC) is used to overcome the long-term catastrophic forgetting issue. The neural pruning acts as long-term depression to prune task-irrelevant parameters, while the novel synaptic consolidation acts as long-term potentiation to strengthen task-relevant parameters. During the training, this confrontation achieves a balance in that only crucial parameters remain, and non-significant parameters are freed to learn subsequent tasks. ANPyC avoids forgetting important information and makes the model efficient to learn a large number of tasks. Specifically, the neural pruning iteratively relaxes the current task's parameter conditions to expand the common parameter subspace of the task; the synaptic consolidation strategy, which consists of a structure-aware parameter-importance measurement and an element-wise parameter updating strategy, decreases the cumulative error when learning new tasks. The full source code is available at https://github.com/GeoX-Lab/ANPyC.","url_abs":"https://arxiv.org/abs/1912.09091v3","url_pdf":"https://arxiv.org/pdf/1912.09091v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"overcoming-long-term-catastrophic-forgetting","repo_url":"https://github.com/GeoX-Lab/ANPyC","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"}],"methods":[{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1912.09091","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1912.09091"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/GeoX-Lab/ANPyC","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"official":{"samples":5,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5d78e1f7a1afb766","entry":"bias_variable","repo":"GeoX-Lab/ANPyC","repo_kind":"official","path":"Permuted_MNIST/ewc/model.py","file_url":"https://github.com/GeoX-Lab/ANPyC/blob/HEAD/Permuted_MNIST/ewc/model.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5d78e1f7a1afb766"}},{"code_sha256_prefix":"58a8b70ac32deb13","entry":"bias_variable","repo":"GeoX-Lab/ANPyC","repo_kind":"official","path":"Permuted_MNIST/ANPyC_wi_NP.py","file_url":"https://github.com/GeoX-Lab/ANPyC/blob/HEAD/Permuted_MNIST/ANPyC_wi_NP.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"58a8b70ac32deb13"}},{"code_sha256_prefix":"61fec95c88135011","entry":"weight_variable","repo":"GeoX-Lab/ANPyC","repo_kind":"official","path":"Permuted_MNIST/ANPyC_wi_NP.py","file_url":"https://github.com/GeoX-Lab/ANPyC/blob/HEAD/Permuted_MNIST/ANPyC_wi_NP.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"61fec95c88135011"}},{"code_sha256_prefix":"ee16754dcbf1a272","entry":"weight_variable","repo":"GeoX-Lab/ANPyC","repo_kind":"official","path":"Permuted_MNIST/SI.py","file_url":"https://github.com/GeoX-Lab/ANPyC/blob/HEAD/Permuted_MNIST/SI.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ee16754dcbf1a272"}},{"code_sha256_prefix":"cb07eb141f5648be","entry":"weight_variable","repo":"GeoX-Lab/ANPyC","repo_kind":"official","path":"Permuted_MNIST/ewc/model.py","file_url":"https://github.com/GeoX-Lab/ANPyC/blob/HEAD/Permuted_MNIST/ewc/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cb07eb141f5648be"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}