{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/trire-a-multi-mechanism-learning-paradigm-for-1","title":"TriRE: A Multi-Mechanism Learning Paradigm for Continual Knowledge Retention and Promotion","arxiv_id":"2310.08217","date":"2023-10-12","proceeding":"NeurIPS 2023 11","authors":["Preetha Vijayan","Prashant Bhat","Elahe Arani","Bahram Zonooz"],"abstract":"Continual learning (CL) has remained a persistent challenge for deep neural networks due to catastrophic forgetting (CF) of previously learned tasks. Several techniques such as weight regularization, experience rehearsal, and parameter isolation have been proposed to alleviate CF. Despite their relative success, these research directions have predominantly remained orthogonal and suffer from several shortcomings, while missing out on the advantages of competing strategies. On the contrary, the brain continually learns, accommodates, and transfers knowledge across tasks by simultaneously leveraging several neurophysiological processes, including neurogenesis, active forgetting, neuromodulation, metaplasticity, experience rehearsal, and context-dependent gating, rarely resulting in CF. Inspired by how the brain exploits multiple mechanisms concurrently, we propose TriRE, a novel CL paradigm that encompasses retaining the most prominent neurons for each task, revising and solidifying the extracted knowledge of current and past tasks, and actively promoting less active neurons for subsequent tasks through rewinding and relearning. Across CL settings, TriRE significantly reduces task interference and surpasses different CL approaches considered in isolation.","url_abs":"https://arxiv.org/abs/2310.08217v1","url_pdf":"https://arxiv.org/pdf/2310.08217v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"trire-a-multi-mechanism-learning-paradigm-for-1","repo_url":"https://github.com/NeurAI-Lab/TriRE","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2310.08217","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.08217"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/NeurAI-Lab/TriRE","reach":null}],"summary":{"ran":2,"unverified":2},"by_repo_kind":{"official":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"a34a68bbb3e6ccdf","entry":"Buffer","repo":"NeurAI-Lab/TriRE","repo_kind":"official","path":"models/trire.py","file_url":"https://github.com/NeurAI-Lab/TriRE/blob/HEAD/models/trire.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a34a68bbb3e6ccdf"}},{"code_sha256_prefix":"49006d05df341095","entry":"create_dense_mask_0","repo":"NeurAI-Lab/TriRE","repo_kind":"official","path":"models/trire.py","file_url":"https://github.com/NeurAI-Lab/TriRE/blob/HEAD/models/trire.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"49006d05df341095"}},{"code_sha256_prefix":"217935831367f94e","entry":"ContinualModel","repo":"NeurAI-Lab/TriRE","repo_kind":"official","path":"models/trire.py","file_url":"https://github.com/NeurAI-Lab/TriRE/blob/HEAD/models/trire.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"217935831367f94e"}},{"code_sha256_prefix":"35d15b3470cd0d36","entry":"TriRE","repo":"NeurAI-Lab/TriRE","repo_kind":"official","path":"models/trire.py","file_url":"https://github.com/NeurAI-Lab/TriRE/blob/HEAD/models/trire.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"35d15b3470cd0d36"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}