{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/task-arithmetic-in-the-tangent-space-improved","title":"Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models","arxiv_id":"2305.12827","date":"2023-05-22","proceeding":"NeurIPS 2023 11","authors":["Guillermo Ortiz-Jimenez","Alessandro Favero","Pascal Frossard"],"abstract":"Task arithmetic has recently emerged as a cost-effective and scalable approach to edit pre-trained models directly in weight space: By adding the fine-tuned weights of different tasks, the model's performance can be improved on these tasks, while negating them leads to task forgetting. Yet, our understanding of the effectiveness of task arithmetic and its underlying principles remains limited. We present a comprehensive study of task arithmetic in vision-language models and show that weight disentanglement is the crucial factor that makes it effective. This property arises during pre-training and manifests when distinct directions in weight space govern separate, localized regions in function space associated with the tasks. Notably, we show that fine-tuning models in their tangent space by linearizing them amplifies weight disentanglement. This leads to substantial performance improvements across multiple task arithmetic benchmarks and diverse models. Building on these findings, we provide theoretical and empirical analyses of the neural tangent kernel (NTK) of these models and establish a compelling link between task arithmetic and the spatial localization of the NTK eigenfunctions. Overall, our work uncovers novel insights into the fundamental mechanisms of task arithmetic and offers a more reliable and effective approach to edit pre-trained models through the NTK linearization.","url_abs":"https://arxiv.org/abs/2305.12827v3","url_pdf":"https://arxiv.org/pdf/2305.12827v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"task-arithmetic-in-the-tangent-space-improved","repo_url":"https://github.com/gortizji/tangent_task_arithmetic","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"disentanglement","task_name":"Disentanglement"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"task-arithmetic","task_name":"Task Arithmetic"}],"methods":[{"method_slug":"ntk","method_name":"NTK"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-classification-on-dtd","task":"Image Classification","dataset":"DTD","model":"Linear FT(ViT-L/14)","rank_in_archive_order":1,"of":11,"metrics":{"Accuracy":"90.0"},"uses_additional_data":true}],"syntology":{"syntology_url":"https://syntology.ai/paper/2305.12827","atlas_url":"https://app.syntology.ai/?focus=2305.12827","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.12827"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gortizji/tangent_task_arithmetic","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"unverified":3},"by_repo_kind":{"official":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2d5d7a12d8de02b7","entry":"cosine_lr","repo":"gortizji/tangent_task_arithmetic","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/gortizji/tangent_task_arithmetic/blob/HEAD/src/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2d5d7a12d8de02b7"}},{"code_sha256_prefix":"aafcff3ad1ccef6e","entry":"accuracy","repo":"gortizji/tangent_task_arithmetic","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/gortizji/tangent_task_arithmetic/blob/HEAD/src/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aafcff3ad1ccef6e"}},{"code_sha256_prefix":"57acc0bee3da475a","entry":"distribute_loader","repo":"gortizji/tangent_task_arithmetic","repo_kind":"official","path":"src/distributed.py","file_url":"https://github.com/gortizji/tangent_task_arithmetic/blob/HEAD/src/distributed.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"57acc0bee3da475a"}},{"code_sha256_prefix":"7d051a7adefedb6c","entry":"torch_load_old","repo":"gortizji/tangent_task_arithmetic","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/gortizji/tangent_task_arithmetic/blob/HEAD/src/utils.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7d051a7adefedb6c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}