{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/reasons-and-solutions-for-the-decline-in","title":"Reasons and Solutions for the Decline in Model Performance after Editing","arxiv_id":"2410.23843","date":"2024-10-31","proceeding":null,"authors":["Xiusheng Huang","Jiaxiang Liu","Yequan Wang","Kang Liu"],"abstract":"Knowledge editing technology has received widespread attention for low-cost updates of incorrect or outdated knowledge in large-scale language models. However, recent research has found that edited models often exhibit varying degrees of performance degradation. The reasons behind this phenomenon and potential solutions have not yet been provided. In order to investigate the reasons for the performance decline of the edited model and optimize the editing method, this work explores the underlying reasons from both data and model perspectives. Specifically, 1) from a data perspective, to clarify the impact of data on the performance of editing models, this paper first constructs a Multi-Question Dataset (MQD) to evaluate the impact of different types of editing data on model performance. The performance of the editing model is mainly affected by the diversity of editing targets and sequence length, as determined through experiments. 2) From a model perspective, this article explores the factors that affect the performance of editing models. The results indicate a strong correlation between the L1-norm of the editing model layer and the editing accuracy, and clarify that this is an important factor leading to the bottleneck of editing performance. Finally, in order to improve the performance of the editing model, this paper further proposes a Dump for Sequence (D4S) method, which successfully overcomes the previous editing bottleneck by reducing the L1-norm of the editing layer, allowing users to perform multiple effective edits and minimizing model damage. Our code is available at https://github.com/nlpkeg/D4S.","url_abs":"https://arxiv.org/abs/2410.23843v1","url_pdf":"https://arxiv.org/pdf/2410.23843v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"reasons-and-solutions-for-the-decline-in","repo_url":"https://github.com/nlpkeg/D4S","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"knowledge-editing","task_name":"knowledge editing"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2410.23843","atlas_url":"https://app.syntology.ai/?focus=2410.23843","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.23843"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nlpkeg/D4S","reach":{"status":"ok"}}],"summary":{"ran_fixture":1,"unverified":2},"by_repo_kind":{"official":{"samples":3,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"bad421a5838188a7","entry":"upd_matrix_match_shape","repo":"nlpkeg/D4S","repo_kind":"official","path":"easyeditor/models/memit/compute_z.py","file_url":"https://github.com/nlpkeg/D4S/blob/HEAD/easyeditor/models/memit/compute_z.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bad421a5838188a7"}},{"code_sha256_prefix":"6e948548b31b79f4","entry":"get_cache","repo":"nlpkeg/D4S","repo_kind":"official","path":"easyeditor/models/memit/memit_main.py","file_url":"https://github.com/nlpkeg/D4S/blob/HEAD/easyeditor/models/memit/memit_main.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6e948548b31b79f4"}},{"code_sha256_prefix":"7e05703117232be7","entry":"get_cov","repo":"nlpkeg/D4S","repo_kind":"official","path":"easyeditor/models/memit/compute_z.py","file_url":"https://github.com/nlpkeg/D4S/blob/HEAD/easyeditor/models/memit/compute_z.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7e05703117232be7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}