{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/memoir-lifelong-model-editing-with-minimal","title":"MEMOIR: Lifelong Model Editing with Minimal Overwrite and Informed Retention for LLMs","arxiv_id":"2506.07899","date":"2025-06-09","proceeding":null,"authors":["Ke Wang","Yiming Qin","Nikolaos Dimitriadis","Alessandro Favero","Pascal Frossard"],"abstract":"Language models deployed in real-world systems often require post-hoc updates to incorporate new or corrected knowledge. However, editing such models efficiently and reliably - without retraining or forgetting previous information - remains a major challenge. Existing methods for lifelong model editing either compromise generalization, interfere with past edits, or fail to scale to long editing sequences. We propose MEMOIR, a novel scalable framework that injects knowledge through a residual memory, i.e., a dedicated parameter module, while preserving the core capabilities of the pre-trained model. By sparsifying input activations through sample-dependent masks, MEMOIR confines each edit to a distinct subset of the memory parameters, minimizing interference among edits. At inference, it identifies relevant edits by comparing the sparse activation patterns of new queries to those stored during editing. This enables generalization to rephrased queries by activating only the relevant knowledge while suppressing unnecessary memory activation for unrelated prompts. Experiments on question answering, hallucination correction, and out-of-distribution generalization benchmarks across LLaMA-3 and Mistral demonstrate that MEMOIR achieves state-of-the-art performance across reliability, generalization, and locality metrics, scaling to thousands of sequential edits with minimal forgetting.","url_abs":"https://arxiv.org/abs/2506.07899v1","url_pdf":"https://arxiv.org/pdf/2506.07899v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"hallucination","task_name":"Hallucination"},{"task_slug":"model-editing","task_name":"Model Editing"},{"task_slug":"out-of-distribution-generalization","task_name":"Out-of-Distribution Generalization"},{"task_slug":"question-answering","task_name":"Question Answering"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2506.07899","atlas_url":"https://app.syntology.ai/?focus=2506.07899","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2506.07899"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/qym7/MEMOIR","reach":null}],"summary":{"ran":2,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9bfbc4c3715c1766","entry":"EarlyStopMeter","repo":"qym7/MEMOIR","repo_kind":"found_in_text","path":"easyeditor/models/memoir/MEMOIR.py","file_url":"https://github.com/qym7/MEMOIR/blob/HEAD/easyeditor/models/memoir/MEMOIR.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9bfbc4c3715c1766"}},{"code_sha256_prefix":"e195d33398cec7fd","entry":"TopHasher","repo":"qym7/MEMOIR","repo_kind":"found_in_text","path":"easyeditor/models/memoir/MEMOIR.py","file_url":"https://github.com/qym7/MEMOIR/blob/HEAD/easyeditor/models/memoir/MEMOIR.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e195d33398cec7fd"}},{"code_sha256_prefix":"dcd59724aa487a06","entry":"MEMOIR","repo":"qym7/MEMOIR","repo_kind":"found_in_text","path":"easyeditor/models/memoir/MEMOIR.py","file_url":"https://github.com/qym7/MEMOIR/blob/HEAD/easyeditor/models/memoir/MEMOIR.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dcd59724aa487a06"}},{"code_sha256_prefix":"919cfbb43215ffca","entry":"MEMOIRAdapter","repo":"qym7/MEMOIR","repo_kind":"found_in_text","path":"easyeditor/models/memoir/MEMOIR.py","file_url":"https://github.com/qym7/MEMOIR/blob/HEAD/easyeditor/models/memoir/MEMOIR.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"919cfbb43215ffca"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}