{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/transformer-patcher-one-mistake-worth-one","title":"Transformer-Patcher: One Mistake worth One Neuron","arxiv_id":"2301.09785","date":"2023-01-24","proceeding":null,"authors":["Zeyu Huang","Yikang Shen","Xiaofeng Zhang","Jie zhou","Wenge Rong","Zhang Xiong"],"abstract":"Large Transformer-based Pretrained Language Models (PLMs) dominate almost all Natural Language Processing (NLP) tasks. Nevertheless, they still make mistakes from time to time. For a model deployed in an industrial environment, fixing these mistakes quickly and robustly is vital to improve user experiences. Previous works formalize such problems as Model Editing (ME) and mostly focus on fixing one mistake. However, the one-mistake-fixing scenario is not an accurate abstraction of the real-world challenge. In the deployment of AI services, there are ever-emerging mistakes, and the same mistake may recur if not corrected in time. Thus a preferable solution is to rectify the mistakes as soon as they appear nonstop. Therefore, we extend the existing ME into Sequential Model Editing (SME) to help develop more practical editing methods. Our study shows that most current ME methods could yield unsatisfying results in this scenario. We then introduce Transformer-Patcher, a novel model editor that can shift the behavior of transformer-based models by simply adding and training a few neurons in the last Feed-Forward Network layer. Experimental results on both classification and generation tasks show that Transformer-Patcher can successively correct up to thousands of errors (Reliability) and generalize to their equivalent inputs (Generality) while retaining the model's accuracy on irrelevant inputs (Locality). Our method outperforms previous fine-tuning and HyperNetwork-based methods and achieves state-of-the-art performance for Sequential Model Editing (SME). The code is available at https://github.com/ZeroYuHuang/Transformer-Patcher.","url_abs":"https://arxiv.org/abs/2301.09785v1","url_pdf":"https://arxiv.org/pdf/2301.09785v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"transformer-patcher-one-mistake-worth-one","repo_url":"https://github.com/zeroyuhuang/transformer-patcher","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"model-editing","task_name":"Model Editing"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2301.09785","atlas_url":"https://app.syntology.ai/?focus=2301.09785","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2301.09785"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/ZeroYuHuang/Transform","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zeroyuhuang/transformer-patcher","reach":null}],"summary":{"ran_fixture":1,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"3ef32b5700dabe8c","entry":"loss_func","repo":"ZeroYuHuang/Transformer-Patcher","repo_kind":"official","path":"src/models/patch.py","file_url":"https://github.com/ZeroYuHuang/Transformer-Patcher/blob/HEAD/src/models/patch.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3ef32b5700dabe8c"}},{"code_sha256_prefix":"d8cd20d238673a86","entry":"top_k_loss_func","repo":"ZeroYuHuang/Transformer-Patcher","repo_kind":"official","path":"src/models/patch.py","file_url":"https://github.com/ZeroYuHuang/Transformer-Patcher/blob/HEAD/src/models/patch.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d8cd20d238673a86"}},{"code_sha256_prefix":"b2149eac0c5ba35a","entry":"ModifyLinearOutput","repo":"zeroyuhuang/transformer-patcher","repo_kind":"official","path":"src/models/patch.py","file_url":"https://github.com/zeroyuhuang/transformer-patcher/blob/HEAD/src/models/patch.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b2149eac0c5ba35a"}},{"code_sha256_prefix":"8b77980c5b963ae8","entry":"list_split","repo":"ZeroYuHuang/Transformer-Patcher","repo_kind":"official","path":"src/models/patch.py","file_url":"https://github.com/ZeroYuHuang/Transformer-Patcher/blob/HEAD/src/models/patch.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8b77980c5b963ae8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}