{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/model-editing-at-scale-leads-to-gradual-and","title":"Model Editing at Scale leads to Gradual and Catastrophic Forgetting","arxiv_id":"2401.07453","date":"2024-01-15","proceeding":null,"authors":["Akshat Gupta","Anurag Rao","Gopala Anumanchipalli"],"abstract":"Editing knowledge in large language models is an attractive capability to have which allows us to correct incorrectly learnt facts during pre-training, as well as update the model with an ever-growing list of new facts. While existing model editing techniques have shown promise, they are usually evaluated using metrics for reliability, specificity and generalization over one or few edits. We argue that for model editing to have practical utility, we must be able to make multiple edits to the same model. With this in mind, we evaluate the current model editing methods at scale, focusing on two state of the art methods: ROME and MEMIT. We find that as the model is edited sequentially with multiple facts, it continually forgets previously edited facts and the ability to perform downstream tasks. This forgetting happens in two phases -- an initial gradual but progressive forgetting phase followed by abrupt or catastrophic forgetting phase. Both gradual and catastrophic forgetting limit the usefulness of model editing methods at scale -- the former making model editing less effective as multiple edits are made to the model while the latter caps the scalability of such model editing methods. Our analysis also highlights other key limitations of ROME and MEMIT at scale. With our work, we push for the development and evaluation of model editing methods keeping scalability in mind.","url_abs":"https://arxiv.org/abs/2401.07453v4","url_pdf":"https://arxiv.org/pdf/2401.07453v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"model-editing-at-scale-leads-to-gradual-and","repo_url":"https://github.com/scalable-model-editing/gradual-catastrophic-forgetting","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"model-editing","task_name":"Model Editing"},{"task_slug":"specificity","task_name":"Specificity"}],"methods":[{"method_slug":"rome","method_name":"ROME"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2401.07453","atlas_url":"https://app.syntology.ai/?focus=2401.07453","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.07453"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/scalable-model-editing/gradual-catastrophic-forgetting","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran_draft_wrong":2,"ran":5,"unverified":2},"by_repo_kind":{"official":{"samples":9,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"27659a9c234ffb11","entry":"binary_log_probs","repo":"scalable-model-editing/gradual-catastrophic-forgetting","repo_kind":"official","path":"baselines/mend/losses.py","file_url":"https://github.com/scalable-model-editing/gradual-catastrophic-forgetting/blob/HEAD/baselines/mend/losses.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"27659a9c234ffb11"}},{"code_sha256_prefix":"920b394e7ad80c53","entry":"hierarchical_subsequence","repo":"scalable-model-editing/gradual-catastrophic-forgetting","repo_kind":"official","path":"util/nethook.py","file_url":"https://github.com/scalable-model-editing/gradual-catastrophic-forgetting/blob/HEAD/util/nethook.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"920b394e7ad80c53"}},{"code_sha256_prefix":"8b02eb3a3733d653","entry":"kl_loc_loss","repo":"scalable-model-editing/gradual-catastrophic-forgetting","repo_kind":"official","path":"baselines/mend/losses.py","file_url":"https://github.com/scalable-model-editing/gradual-catastrophic-forgetting/blob/HEAD/baselines/mend/losses.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8b02eb3a3733d653"}},{"code_sha256_prefix":"6a33727392194b2c","entry":"moving_average","repo":"scalable-model-editing/gradual-catastrophic-forgetting","repo_kind":"official","path":"downstream_eval/current_edit_scores.py","file_url":"https://github.com/scalable-model-editing/gradual-catastrophic-forgetting/blob/HEAD/downstream_eval/current_edit_scores.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6a33727392194b2c"}},{"code_sha256_prefix":"8ccc8886c554a1ce","entry":"multiclass_log_probs","repo":"scalable-model-editing/gradual-catastrophic-forgetting","repo_kind":"official","path":"baselines/mend/losses.py","file_url":"https://github.com/scalable-model-editing/gradual-catastrophic-forgetting/blob/HEAD/baselines/mend/losses.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8ccc8886c554a1ce"}},{"code_sha256_prefix":"70f6ab8bde55420e","entry":"recursive_copy","repo":"scalable-model-editing/gradual-catastrophic-forgetting","repo_kind":"official","path":"util/nethook.py","file_url":"https://github.com/scalable-model-editing/gradual-catastrophic-forgetting/blob/HEAD/util/nethook.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"70f6ab8bde55420e"}},{"code_sha256_prefix":"440ff98c2b1ae1aa","entry":"subsequence","repo":"scalable-model-editing/gradual-catastrophic-forgetting","repo_kind":"official","path":"util/nethook.py","file_url":"https://github.com/scalable-model-editing/gradual-catastrophic-forgetting/blob/HEAD/util/nethook.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"440ff98c2b1ae1aa"}},{"code_sha256_prefix":"d9a5bcaef451668f","entry":"get_model","repo":"scalable-model-editing/gradual-catastrophic-forgetting","repo_kind":"official","path":"baselines/mend/models.py","file_url":"https://github.com/scalable-model-editing/gradual-catastrophic-forgetting/blob/HEAD/baselines/mend/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d9a5bcaef451668f"}},{"code_sha256_prefix":"87e77d8c9c3620e1","entry":"get_tokenizer","repo":"scalable-model-editing/gradual-catastrophic-forgetting","repo_kind":"official","path":"baselines/mend/models.py","file_url":"https://github.com/scalable-model-editing/gradual-catastrophic-forgetting/blob/HEAD/baselines/mend/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"87e77d8c9c3620e1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}