{"url":"/task/task-arithmetic","name":"Task Arithmetic","slug":"task-arithmetic","description_markdown":"A task vector specifies a direction in the weight space of a pre-trained\r\nmodel, such that movement in that direction improves performance on the task. We build task vectors by subtracting the weights of a pre-trained model from the weights of the same model after fine-tuning on a task. We show that these task vectors can be modified and combined together through arithmetic operations such as negation and addition, and the behavior of the resulting model is steered accordingly.","categories":[{"name":"Methodology","url":"/area/methodology"}],"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","slug_source":"derived"},"counts":{"papers_tagged":61,"papers_with_code":24,"benchmarks":0,"benchmark_tables_in_archive":0,"benchmark_tables_shown":0,"benchmark_tables_withheld_as_spam":0,"benchmark_definition":"a leaderboard table with at least one row; benchmark_tables_shown also counts the zero-row tables; benchmark_tables_in_archive adds the tables withheld as spam","datasets":0,"subtasks":0,"parent_tasks":1},"benchmarks":[],"datasets":[],"subtasks":[],"parent_tasks":[{"url":"/task/multi-task-learning","name":"Multi-Task Learning"}],"papers":{"order":"repositories listed in the archive (desc), then date (desc); the archive holds no stars","population":"papers tagged with this task that list at least one repository in the archive","shown":24,"of":24,"tagged_in_all":61,"items":[{"url":"/paper/editing-models-with-task-arithmetic","title":"Editing Models with Task Arithmetic","date":"2022-12-08","arxiv_id":"2212.04089","repositories_listed":9,"syntology":{"n":15,"n_ran":5,"n_unverified":10,"n_pointer_only":4}},{"url":"/paper/language-models-are-homer-simpson-safety-re","title":"Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic","date":"2024-02-19","arxiv_id":"2402.11746","repositories_listed":3,"syntology":{"n":16,"n_ran":9,"n_unverified":7,"n_pointer_only":7}},{"url":"/paper/localize-and-stitch-efficient-model-merging","title":"Localize-and-Stitch: Efficient Model Merging via Sparse Task Arithmetic","date":"2024-08-24","arxiv_id":"2408.13656","repositories_listed":2,"syntology":{"n":7,"n_ran":7,"n_unverified":0,"n_pointer_only":0}},{"url":"/paper/fine-tuning-linear-layers-only-is-a-simple","title":"Fine-Tuning Attention Modules Only: Enhancing Weight Disentanglement in Task Arithmetic","date":"2024-07-09","arxiv_id":"2407.07089","repositories_listed":2,"syntology":{"n":5,"n_ran":3,"n_unverified":2,"n_pointer_only":0}},{"url":"/paper/localizing-task-information-for-improved","title":"Localizing Task Information for Improved Model Merging and Compression","date":"2024-05-13","arxiv_id":"2405.07813","repositories_listed":2,"syntology":{"n":5,"n_ran":3,"n_unverified":2,"n_pointer_only":0}},{"url":"/paper/calm-consensus-aware-localized-merging-for","title":"CALM: Consensus-Aware Localized Merging for Multi-Task Learning","date":"2025-06-16","arxiv_id":"2506.13406","repositories_listed":1,"syntology":{"n":10,"n_ran":3,"n_unverified":7,"n_pointer_only":10}},{"url":"/paper/cross-model-transfer-of-task-vectors-via-few","title":"Cross-Model Transfer of Task Vectors via Few-Shot Orthogonal Alignment","date":"2025-05-17","arxiv_id":"2505.12021","repositories_listed":1,"syntology":null},{"url":"/paper/investigating-task-arithmetic-for-zero-shot","title":"Investigating Task Arithmetic for Zero-Shot Information Retrieval","date":"2025-05-01","arxiv_id":"2505.00649","repositories_listed":1,"syntology":null},{"url":"/paper/leveraging-submodule-linearity-enhances-task","title":"Leveraging Submodule Linearity Enhances Task Arithmetic Performance in LLMs","date":"2025-04-15","arxiv_id":"2504.10902","repositories_listed":1,"syntology":{"n":1,"n_ran":1,"n_unverified":0,"n_pointer_only":1}},{"url":"/paper/efficient-model-editing-with-task-localized","title":"Efficient Model Editing with Task-Localized Sparse Fine-tuning","date":"2025-04-03","arxiv_id":"2504.02620","repositories_listed":1,"syntology":null},{"url":"/paper/efficient-model-editing-with-task-vector","title":"Efficient Model Editing with Task Vector Bases: A Theoretical Framework and Scalable Approach","date":"2025-02-03","arxiv_id":"2502.01015","repositories_listed":1,"syntology":{"n":4,"n_ran":1,"n_unverified":3,"n_pointer_only":0}},{"url":"/paper/task-singular-vectors-reducing-task","title":"Task Singular Vectors: Reducing Task Interference in Model Merging","date":"2024-11-26","arxiv_id":"2412.00081","repositories_listed":1,"syntology":null},{"url":"/paper/negmerge-consensual-weight-negation-for","title":"NegMerge: Consensual Weight Negation for Strong Machine Unlearning","date":"2024-10-08","arxiv_id":"2410.05583","repositories_listed":1,"syntology":{"n":6,"n_ran":3,"n_unverified":3,"n_pointer_only":6}},{"url":"/paper/towards-diverse-device-heterogeneous","title":"Towards Diverse Device Heterogeneous Federated Learning via Task Arithmetic Knowledge Integration","date":"2024-09-27","arxiv_id":"2409.18461","repositories_listed":1,"syntology":null},{"url":"/paper/knowledge-composition-using-task-vectors-with","title":"Knowledge Composition using Task Vectors with Learned Anisotropic Scaling","date":"2024-07-03","arxiv_id":"2407.02880","repositories_listed":1,"syntology":{"n":4,"n_ran":3,"n_unverified":1,"n_pointer_only":4}},{"url":"/paper/no-train-but-gain-language-arithmetic-for","title":"No Train but Gain: Language Arithmetic for training-free Language Adapters enhancement","date":"2024-04-24","arxiv_id":"2404.15737","repositories_listed":1,"syntology":null},{"url":"/paper/have-you-merged-my-model-on-the-robustness-of","title":"Have You Merged My Model? On The Robustness of Large Language Model IP Protection Methods Against Model Merging","date":"2024-04-08","arxiv_id":"2404.05188","repositories_listed":1,"syntology":{"n":1,"n_ran":1,"n_unverified":0,"n_pointer_only":1}},{"url":"/paper/merging-multi-task-models-via-weight","title":"Merging Multi-Task Models via Weight-Ensembling Mixture of Experts","date":"2024-02-01","arxiv_id":"2402.00433","repositories_listed":1,"syntology":{"n":1,"n_ran":1,"n_unverified":0,"n_pointer_only":1}},{"url":"/paper/concrete-subspace-learning-based-interference","title":"Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion","date":"2023-12-11","arxiv_id":"2312.06173","repositories_listed":1,"syntology":{"n":9,"n_ran":8,"n_unverified":1,"n_pointer_only":9}},{"url":"/paper/model-merging-by-uncertainty-based-gradient","title":"Model Merging by Uncertainty-Based Gradient Matching","date":"2023-10-19","arxiv_id":"2310.12808","repositories_listed":1,"syntology":{"n":2,"n_ran":2,"n_unverified":0,"n_pointer_only":2}},{"url":"/paper/parameter-efficient-multi-task-model-fusion","title":"Parameter Efficient Multi-task Model Fusion with Partial Linearization","date":"2023-10-07","arxiv_id":"2310.04742","repositories_listed":1,"syntology":{"n":4,"n_ran":3,"n_unverified":1,"n_pointer_only":4}},{"url":"/paper/adamerging-adaptive-model-merging-for-multi","title":"AdaMerging: Adaptive Model Merging for Multi-Task Learning","date":"2023-10-04","arxiv_id":"2310.02575","repositories_listed":1,"syntology":{"n":3,"n_ran":3,"n_unverified":0,"n_pointer_only":0}},{"url":"/paper/task-arithmetic-in-the-tangent-space-improved","title":"Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models","date":"2023-05-22","arxiv_id":"2305.12827","repositories_listed":1,"syntology":{"n":4,"n_ran":1,"n_unverified":3,"n_pointer_only":0}},{"url":"/paper/an-empirical-study-of-multimodal-model","title":"An Empirical Study of Multimodal Model Merging","date":"2023-04-28","arxiv_id":"2304.14933","repositories_listed":1,"syntology":{"n":1,"n_ran":1,"n_unverified":0,"n_pointer_only":1}}],"syntology_records":18,"syntology_note":"a paper without a record is not a recorded non-run: it may lack an arXiv id or simply be absent from the graph layer"},"description_links":{"kept":0,"unwrapped_to_text":0,"bare_urls_linked":0,"relative_images_dropped":0,"rule":"internal links are kept only when the target slug exists in the catalog"},"syntology":{"read_at":"2026-09-24T18:15:14+00:00","claim":"Per-sample execution status on synthesized fixtures ('ran N of M samples'); not a correctness claim and not a ranking signal.","status_vocabulary":{"ran_honours":"ran, honoured the contract we drafted","ran_violates":"ran, violated the contract we drafted","ran_draft_wrong":"ran; our contract draft was wrong, not the code","ran_fixture":"ran; our fixture could not drive it","ran":"ran on a synthesized input","unverified":"unverified (harvested, no recorded run)"}},"not_shown":{"libraries":"the archive has no per-task library table","trend_sparklines":"the Trend column of the benchmarks table was a rendered image; it is not in the archive","social_and_latest_sorts":"stars and social signals are not in the archive"}}