Browse State-of-the-Art › Task Arithmetic
Task Arithmetic
24 papers with code · 0 benchmarks · 0 datasets archive 2025-07-28
A task vector specifies a direction in the weight space of a pre-trained model, such that movement in that direction improves performance on the task. We build task vectors by subtracting the weights of a pre-trained model from the weights of the same model after fine-tuning on a task. We show that these task vectors can be modified and combined together through arithmetic operations such as negation and addition, and the behavior of the resulting model is steered accordingly.
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
No benchmark for this task in the archive.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
No dataset record in the archive lists this task.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 61 · with a code link: 24 · where Syntology ran a sample: 18 (14 with a run with no instrument failure, 4 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (18 of 61 tagged: 14 with a run with no instrument failure, 4 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 61 of 61, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
8 Dec 2022 9 repositories listed Syntology official: no sample here; runs from other or unrecorded repositories · 5 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 10 unverified (of 15 harvested samples) · 4 pointer-only (licence)Changing how pre-trained models behave -- e. g., improving their performance on a downstream task or mitigating biases learned during pre-training -- is a common practice when developing machine learning systems.
-
19 Feb 2024 3 repositories listed Syntology official (archive's flag): 6 ran · 13 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 1 violated, 0 with no contract checked; 11 where Syntology's instrument failed) · 3 unverified (of 16 harvested samples) · 9 pointer-only (licence)We demonstrate the effectiveness of RESTA in both parameter-efficient and full fine-tuning, covering a wide range of downstream tasks, including instruction following in Chinese, English, and Hindi, as well as…
-
24 Aug 2024 2 repositories listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 5 where Syntology's instrument failed) · 0 unverified (of 7 harvested samples)Our algorithm works in two steps: i) Localization: identify tiny (1% of the total parameters) localized regions in the finetuned models containing essential skills for the downstream tasks, and ii) Stitching…
-
9 Jul 2024 2 repositories listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 5 harvested samples)To further understand how our method improves the weight disentanglement of task arithmetic, we present a comprehensive study of task arithmetic by differentiating the role of the representation module and task-specific…
-
13 May 2024 2 repositories listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 5 harvested samples) · 4 pointer-only (licence)For this reason, we propose Consensus Merging, an algorithm that eliminates such weights and improves the general performance of existing model merging approaches.
-
16 Jun 2025 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 4 unverified (of 10 harvested samples) · 10 pointer-only (licence)Model merging aims to integrate the strengths of multiple fine-tuned models into a unified model while preserving task-specific capabilities.
-
17 May 2025 1 repository listedTo address this challenge, we propose a method based on few-shot orthogonal alignment, which aligns task vectors to the parameter space of a differently pre-trained target model.
-
1 May 2025 1 repository listedIn addition to these empirical gains, our analysis provides insights into the strengths and limitations of Task Arithmetic as a practical strategy for zero-shot learning and model adaptation.
-
15 Apr 2025 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)Task arithmetic is a straightforward yet highly effective strategy for model merging, enabling the resultant model to exhibit multi-task capabilities.
-
3 Apr 2025 1 repository listedHowever, existing methods rely on network linearization to derive task vectors, leading to computational bottlenecks during training and inference.
-
3 Feb 2025 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 4 harvested samples) · 2 pointer-only (licence)Task vectors, which are derived from the difference between pre-trained and fine-tuned model weights, enable flexible task adaptation and model merging through arithmetic operations such as addition and negation.
-
26 Nov 2024 1 repository listedIn this paper, we study task vectors at the layer level, focusing on task layer matrices and their singular value decomposition.
-
8 Oct 2024 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified (of 6 harvested samples) · 6 pointer-only (licence)Current methods, such as task arithmetic, rely on fine-tuning models on the forget set, generating a task vector, and subtracting it from the original model.
-
27 Sep 2024 1 repository listedTo address these challenges, we introduce TAKFL, a novel KD-based framework that treats the knowledge transfer from each device prototype's ensemble as a separate task, independently distilling each to preserve its…
-
3 Jul 2024 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples) · 4 pointer-only (licence)In particular, we show that (1) learned anisotropic scaling allows task vectors to be more disentangled, causing less interference in composition; (2) task vector composition excels with scarce or no labeled data and is…
-
24 Apr 2024 1 repository listedModular deep learning is the state-of-the-art solution for lifting the curse of multilinguality, preventing the impact of negative interference and enabling cross-lingual performance in Multilingual Pre-trained Language…
-
8 Apr 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)Model merging is a promising lightweight model empowerment technique that does not rely on expensive computing devices (e. g., GPUs) or require the collection of specific training data.
-
1 Feb 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)A notable challenge is mitigating the interference between parameters of different models, which can substantially deteriorate performance.
-
11 Dec 2023 1 repository listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 0 violated, 4 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 9 harvested samples) · 9 pointer-only (licence)At the upper level, we focus on learning a shared Concrete mask to identify the subspace, while at the inner level, model merging is performed to maximize the performance of the merged model.
-
19 Oct 2023 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)Models trained on different datasets can be merged by a weighted-averaging of their parameters, but why does it work and when can it fail?
-
7 Oct 2023 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples) · 4 pointer-only (licence)We demonstrate that our partial linearization technique enables a more effective fusion of multiple tasks into a single model, outperforming standard adapter tuning and task arithmetic alone.
-
4 Oct 2023 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)This approach aims to autonomously learn the coefficients for model merging, either in a task-wise or layer-wise manner, without relying on the original training data.
-
22 May 2023 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 4 harvested samples) · 3 pointer-only (licence)Task arithmetic has recently emerged as a cost-effective and scalable approach to edit pre-trained models directly in weight space: By adding the fine-tuned weights of different tasks, the model's performance can be…
-
28 Apr 2023 1 repository listed Syntology official: no sample here; runs from other or unrecorded repositories · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)In this paper, we expand on this concept to a multimodal setup by merging transformers trained on different modalities.
-
Transferring Visual Explainability of Self-Explaining Models through Task Arithmetic6 Jul 2025 0 repositories listedIn scenarios requiring both prediction and explanation efficiency for image classification, self-explaining models that perform both tasks in a single inference are effective.
-
DuET: Dual Incremental Object Detection via Exemplar-Free Task Arithmetic26 Jun 2025 0 repositories listedReal-world object detection systems, such as those in autonomous driving and surveillance, must continuously learn new object categories and simultaneously adapt to changing environmental conditions.
-
CultureMERT: Continual Pre-Training for Cross-Cultural Music Representation Learning21 Jun 2025 0 repositories listedRecent advances in music foundation models have improved audio representation learning, yet their effectiveness across diverse musical traditions remains limited.
-
Subspace-Boosted Model Merging19 Jun 2025 0 repositories listedModel merging enables the combination of multiple specialized expert models into a single model capable of performing multiple tasks.
-
FedRPCA: Enhancing Federated LoRA Aggregation Using Robust PCA1 Jun 2025 0 repositories listedLoRA has emerged as one of the most promising fine-tuning techniques, especially for federated learning (FL), since it significantly reduces communication and computation costs at resource-constrained clients.
-
On Fairness of Task Arithmetic: The Role of Task Vectors30 May 2025 0 repositories listedModel editing techniques, particularly task arithmetic using task vectors, have shown promise in efficiently modifying pre-trained models through arithmetic operations like task addition and negation.
-
Scalable Strategies for Continual Learning with Replay18 May 2025 0 repositories listed
-
MCU: Improving Machine Unlearning through Mode Connectivity16 May 2025 0 repositories listed
-
CAT Merging: A Training-Free Approach for Resolving Conflicts in Model Merging11 May 2025 0 repositories listed
-
Single-Input Multi-Output Model Merging: Leveraging Foundation Models for Dense Multi-Task Learning15 Apr 2025 0 repositories listed
-
When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers15 Apr 2025 0 repositories listed
-
OpenThaiGPT 1.6 and R1: Thai-Centric Open Source and Reasoning Large Language Models2 Apr 2025 0 repositories listed
-
Disentangling Task Interference within Neurons: Model Merging in Alignment with Neuronal Mechanisms7 Mar 2025 0 repositories listed
-
Layer-Aware Task Arithmetic: Disentangling Task-Specific and Instruction-Following Knowledge27 Feb 2025 0 repositories listed
-
Neural Networks Remember More: The Power of Parameter Isolation and Combination16 Feb 2025 0 repositories listed
-
Mediator: Memory-efficient LLM Merging with Less Parameter Conflicts and Uncertainty Based Routing6 Feb 2025 0 repositories listed
-
Task Arithmetic in Trust Region: A Training-Free Model Merging Approach to Navigate Knowledge Conflicts25 Jan 2025 0 repositories listed
-
Soup to go: mitigating forgetting during continual learning with model averaging9 Jan 2025 0 repositories listed
-
BADTV: Unveiling Backdoor Threats in Third-Party Task Vectors4 Jan 2025 0 repositories listed
-
When Domain Generalization meets Generalized Category Discovery: An Adaptive Task-Arithmetic Driven Approach1 Jan 2025 0 repositories listed
-
Bias Vector: Mitigating Biases in Language Models with Task Arithmetic Approach16 Dec 2024 0 repositories listed
-
Multi-Task Model Merging via Adaptive Weight Disentanglement27 Nov 2024 0 repositories listed
-
Task Arithmetic Through The Lens Of One-Shot Federated Learning27 Nov 2024 0 repositories listed
-
Beyond Task Vectors: Selective Task Arithmetic Based on Importance Metrics25 Nov 2024 0 repositories listed
-
ATM: Improving Model Merging by Alternating Tuning and Merging5 Nov 2024 0 repositories listed
-
Efficient and Effective Weight-Ensembling Mixture of Experts for Multi-Task Model Merging29 Oct 2024 0 repositories listed
-
The Non-Local Model Merging Problem: Permutation Symmetries and Variance Collapse16 Oct 2024 0 repositories listed
-
What Matters for Model Merging at Scale?4 Oct 2024 0 repositories listed
-
Task Arithmetic for Language Expansion in Speech Translation17 Sep 2024 0 repositories listed
-
MetaGPT: Merging Large Language Models Using Model Exclusive Task Arithmetic17 Jun 2024 0 repositories listed
-
On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion17 Jun 2024 0 repositories listed
-
Task Arithmetic can Mitigate Synthetic-to-Real Gap in Automatic Speech Recognition5 Jun 2024 0 repositories listed
-
HPE-CogVLM: Advancing Vision Language Models with a Head Pose Grounding Task4 Jun 2024 0 repositories listed
-
To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Models6 May 2024 0 repositories listed
-
Ethos: Rectifying Language Models in Orthogonal Parameter Space13 Mar 2024 0 repositories listed
-
Language and Task Arithmetic with Parameter-Efficient Layers for Zero-Shot Summarization15 Nov 2023 0 repositories listed
-
Task Arithmetic with LoRA for Continual Learning4 Nov 2023 0 repositories listed
Syntology lines on 18 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections