{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/consistency-is-the-key-to-further-mitigating","title":"Consistency is the key to further mitigating catastrophic forgetting in continual learning","arxiv_id":"2207.04998","date":"2022-07-11","proceeding":null,"authors":["Prashant Bhat","Bahram Zonooz","Elahe Arani"],"abstract":"Deep neural networks struggle to continually learn multiple sequential tasks due to catastrophic forgetting of previously learned tasks. Rehearsal-based methods which explicitly store previous task samples in the buffer and interleave them with the current task samples have proven to be the most effective in mitigating forgetting. However, Experience Replay (ER) does not perform well under low-buffer regimes and longer task sequences as its performance is commensurate with the buffer size. Consistency in predictions of soft-targets can assist ER in preserving information pertaining to previous tasks better as soft-targets capture the rich similarity structure of the data. Therefore, we examine the role of consistency regularization in ER framework under various continual learning scenarios. We also propose to cast consistency regularization as a self-supervised pretext task thereby enabling the use of a wide variety of self-supervised learning methods as regularizers. While simultaneously enhancing model calibration and robustness to natural corruptions, regularizing consistency in predictions results in lesser forgetting across all continual learning scenarios. Among the different families of regularizers, we find that stricter consistency constraints preserve previous task information in ER better.","url_abs":"https://arxiv.org/abs/2207.04998v1","url_pdf":"https://arxiv.org/pdf/2207.04998v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"consistency-is-the-key-to-further-mitigating","repo_url":"https://github.com/neurai-lab/consistencycl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"}],"methods":[{"method_slug":"experience-replay","method_name":"Experience Replay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2207.04998","atlas_url":"https://app.syntology.ai/?focus=2207.04998","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2207.04998"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/neurai-lab/consistencycl","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"ran_fixture":1,"unverified":2},"by_repo_kind":{"official":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4c2989ace7c5c0da","entry":"conv3x3","repo":"neurai-lab/consistencycl","repo_kind":"official","path":"backbone/ResNet18.py","file_url":"https://github.com/neurai-lab/consistencycl/blob/HEAD/backbone/ResNet18.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4c2989ace7c5c0da"}},{"code_sha256_prefix":"3e30d88eaef01190","entry":"off_diagonal","repo":"neurai-lab/consistencycl","repo_kind":"official","path":"models/cr.py","file_url":"https://github.com/neurai-lab/consistencycl/blob/HEAD/models/cr.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3e30d88eaef01190"}},{"code_sha256_prefix":"5eb0a5255bcf3849","entry":"get_resnet18","repo":"neurai-lab/consistencycl","repo_kind":"official","path":"backbone/ResNet18.py","file_url":"https://github.com/neurai-lab/consistencycl/blob/HEAD/backbone/ResNet18.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5eb0a5255bcf3849"}},{"code_sha256_prefix":"5b5d3ed9fc440f9e","entry":"resnet18","repo":"neurai-lab/consistencycl","repo_kind":"official","path":"backbone/ResNet18.py","file_url":"https://github.com/neurai-lab/consistencycl/blob/HEAD/backbone/ResNet18.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5b5d3ed9fc440f9e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}