{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/efficient-model-editing-with-task-vector","title":"Efficient Model Editing with Task Vector Bases: A Theoretical Framework and Scalable Approach","arxiv_id":"2502.01015","date":"2025-02-03","proceeding":null,"authors":["Siqi Zeng","Yifei He","Weiqiu You","Yifan Hao","Yao-Hung Hubert Tsai","Makoto Yamada","Han Zhao"],"abstract":"Task vectors, which are derived from the difference between pre-trained and fine-tuned model weights, enable flexible task adaptation and model merging through arithmetic operations such as addition and negation. However, existing approaches often rely on heuristics with limited theoretical support, often leading to performance gaps comparing to direct task fine tuning. Meanwhile, although it is easy to manipulate saved task vectors with arithmetic for different purposes, such compositional flexibility demands high memory usage, especially when dealing with a huge number of tasks, limiting scalability. This work addresses these issues with a theoretically grounded framework that explains task vector arithmetic and introduces the task vector bases framework. Building upon existing task arithmetic literature, our method significantly reduces the memory cost for downstream arithmetic with little effort, while achieving competitive performance and maintaining compositional advantage, providing a practical solution for large-scale task arithmetic.","url_abs":"https://arxiv.org/abs/2502.01015v1","url_pdf":"https://arxiv.org/pdf/2502.01015v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficient-model-editing-with-task-vector","repo_url":"https://github.com/uiuctml/TaskVectorBasis","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"model-editing","task_name":"Model Editing"},{"task_slug":"negation","task_name":"Negation"},{"task_slug":"task-arithmetic","task_name":"Task Arithmetic"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2502.01015","atlas_url":"https://app.syntology.ai/?focus=2502.01015","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.01015"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/uiuctml/TaskVectorBasis","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"unverified":2},"by_repo_kind":{"official":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2d5d7a12d8de02b7","entry":"cosine_lr","repo":"uiuctml/TaskVectorBasis","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/uiuctml/TaskVectorBasis/blob/HEAD/src/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2d5d7a12d8de02b7"}},{"code_sha256_prefix":"c2d8826b25837fd9","entry":"get_eval_datasets","repo":"uiuctml/TaskVectorBasis","repo_kind":"official","path":"src/args.py","file_url":"https://github.com/uiuctml/TaskVectorBasis/blob/HEAD/src/args.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c2d8826b25837fd9"}},{"code_sha256_prefix":"aafcff3ad1ccef6e","entry":"accuracy","repo":"uiuctml/TaskVectorBasis","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/uiuctml/TaskVectorBasis/blob/HEAD/src/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aafcff3ad1ccef6e"}},{"code_sha256_prefix":"7d051a7adefedb6c","entry":"torch_load_old","repo":"uiuctml/TaskVectorBasis","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/uiuctml/TaskVectorBasis/blob/HEAD/src/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7d051a7adefedb6c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}