{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/memory-aware-synapses-learning-what-not-to","title":"Memory Aware Synapses: Learning what (not) to forget","arxiv_id":"1711.09601","date":"2017-11-27","proceeding":"ECCV 2018 9","authors":["Rahaf Aljundi","Francesca Babiloni","Mohamed Elhoseiny","Marcus Rohrbach","Tinne Tuytelaars"],"abstract":"Humans can learn in a continuous manner. Old rarely utilized knowledge can be\noverwritten by new incoming information while important, frequently used\nknowledge is prevented from being erased. In artificial learning systems,\nlifelong learning so far has focused mainly on accumulating knowledge over\ntasks and overcoming catastrophic forgetting. In this paper, we argue that,\ngiven the limited model capacity and the unlimited new information to be\nlearned, knowledge has to be preserved or erased selectively. Inspired by\nneuroplasticity, we propose a novel approach for lifelong learning, coined\nMemory Aware Synapses (MAS). It computes the importance of the parameters of a\nneural network in an unsupervised and online manner. Given a new sample which\nis fed to the network, MAS accumulates an importance measure for each parameter\nof the network, based on how sensitive the predicted output function is to a\nchange in this parameter. When learning a new task, changes to important\nparameters can then be penalized, effectively preventing important knowledge\nrelated to previous tasks from being overwritten. Further, we show an\ninteresting connection between a local version of our method and Hebb's\nrule,which is a model for the learning process in the brain. We test our method\non a sequence of object recognition tasks and on the challenging problem of\nlearning an embedding for predicting $<$subject, predicate, object$>$ triplets.\nWe show state-of-the-art performance and, for the first time, the ability to\nadapt the importance of the parameters based on unlabeled data towards what the\nnetwork needs (not) to forget, which may vary depending on test conditions.","url_abs":"http://arxiv.org/abs/1711.09601v4","url_pdf":"http://arxiv.org/pdf/1711.09601v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"memory-aware-synapses-learning-what-not-to","repo_url":"https://github.com/wannabeOG/MAS-PyTorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"memory-aware-synapses-learning-what-not-to","repo_url":"https://github.com/ContinualAI/avalanche","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"memory-aware-synapses-learning-what-not-to","repo_url":"https://github.com/rahafaljundi/MAS-Memory-Aware-Synapses","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"lifelong-learning","task_name":"Lifelong learning"},{"task_slug":"object-recognition","task_name":"Object Recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1711.09601","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1711.09601"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ContinualAI/avalanche","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rahafaljundi/MAS-Memory-Aware-Synapses","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wannabeOG/MAS-PyTorch","reach":null}],"summary":{"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"listed":{"samples":3,"ran":1,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"b8f0a2fb31ee439e","entry":"exp_lr_scheduler","repo":"rahafaljundi/MAS-Memory-Aware-Synapses","repo_kind":"listed","path":"MAS_to_be_published/MAS.py","file_url":"https://github.com/rahafaljundi/MAS-Memory-Aware-Synapses/blob/HEAD/MAS_to_be_published/MAS.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b8f0a2fb31ee439e"}},{"code_sha256_prefix":"842330eb6620cdd5","entry":"compute_forgetting","repo":"wannabeOG/MAS-PyTorch","repo_kind":"listed","path":"mas.py","file_url":"https://github.com/wannabeOG/MAS-PyTorch/blob/HEAD/mas.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"842330eb6620cdd5"}},{"code_sha256_prefix":"c521f897b65d55d6","entry":"mas_train","repo":"wannabeOG/MAS-PyTorch","repo_kind":"listed","path":"mas.py","file_url":"https://github.com/wannabeOG/MAS-PyTorch/blob/HEAD/mas.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"c521f897b65d55d6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}