Task Arithmetic

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

No dataset record in the archive lists this task.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Parent tasks archive 2025-07-28

Most implemented papers archive 2025-07-28

24 shown of 24 papers with code (61 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Editing Models with Task Arithmetic
  8 Dec 2022 9 repositories listed Syntology ran 5 of 15 samples · 10 unverified · 4 pointer-only (licence)
  Changing how pre-trained models behave -- e.
• Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic
  19 Feb 2024 3 repositories listed Syntology ran 9 of 16 samples · 7 unverified · 7 pointer-only (licence)
  We demonstrate the effectiveness of RESTA in both parameter-efficient and full fine-tuning, covering a wide range of downstream tasks, including instruction following in Chinese, English, and Hindi, as well as…
• Localize-and-Stitch: Efficient Model Merging via Sparse Task Arithmetic
  24 Aug 2024 2 repositories listed Syntology ran 7 of 7 samples · 0 unverified
  Our algorithm works in two steps: i) Localization: identify tiny (1% of the total parameters) localized regions in the finetuned models containing essential skills for the downstream tasks, and ii) Stitching:…
• Fine-Tuning Attention Modules Only: Enhancing Weight Disentanglement in Task Arithmetic
  9 Jul 2024 2 repositories listed Syntology ran 3 of 5 samples · 2 unverified
  To further understand how our method improves the weight disentanglement of task arithmetic, we present a comprehensive study of task arithmetic by differentiating the role of the representation module and task-specific…
• Localizing Task Information for Improved Model Merging and Compression
  13 May 2024 2 repositories listed Syntology ran 3 of 5 samples · 2 unverified
  For this reason, we propose Consensus Merging, an algorithm that eliminates such weights and improves the general performance of existing model merging approaches.
• CALM: Consensus-Aware Localized Merging for Multi-Task Learning
  16 Jun 2025 1 repository listed Syntology ran 3 of 10 samples · 7 unverified · 10 pointer-only (licence)
  Model merging aims to integrate the strengths of multiple fine-tuned models into a unified model while preserving task-specific capabilities.
• Cross-Model Transfer of Task Vectors via Few-Shot Orthogonal Alignment
  17 May 2025 1 repository listed
  To address this challenge, we propose a method based on few-shot orthogonal alignment, which aligns task vectors to the parameter space of a differently pre-trained target model.
• Investigating Task Arithmetic for Zero-Shot Information Retrieval
  1 May 2025 1 repository listed
  In addition to these empirical gains, our analysis provides insights into the strengths and limitations of Task Arithmetic as a practical strategy for zero-shot learning and model adaptation.
• Leveraging Submodule Linearity Enhances Task Arithmetic Performance in LLMs
  15 Apr 2025 1 repository listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  Task arithmetic is a straightforward yet highly effective strategy for model merging, enabling the resultant model to exhibit multi-task capabilities.
• Efficient Model Editing with Task-Localized Sparse Fine-tuning
  3 Apr 2025 1 repository listed
  However, existing methods rely on network linearization to derive task vectors, leading to computational bottlenecks during training and inference.
• Efficient Model Editing with Task Vector Bases: A Theoretical Framework and Scalable Approach
  3 Feb 2025 1 repository listed Syntology ran 1 of 4 samples · 3 unverified
  Task vectors, which are derived from the difference between pre-trained and fine-tuned model weights, enable flexible task adaptation and model merging through arithmetic operations such as addition and negation.
• Task Singular Vectors: Reducing Task Interference in Model Merging
  26 Nov 2024 1 repository listed
  In this paper, we study task vectors at the layer level, focusing on task layer matrices and their singular value decomposition.
• NegMerge: Consensual Weight Negation for Strong Machine Unlearning
  8 Oct 2024 1 repository listed Syntology ran 3 of 6 samples · 3 unverified · 6 pointer-only (licence)
  Current methods, such as task arithmetic, rely on fine-tuning models on the forget set, generating a task vector, and subtracting it from the original model.
• Towards Diverse Device Heterogeneous Federated Learning via Task Arithmetic Knowledge Integration
  27 Sep 2024 1 repository listed
  To address these challenges, we introduce TAKFL, a novel KD-based framework that treats the knowledge transfer from each device prototype's ensemble as a separate task, independently distilling each to preserve its…
• Knowledge Composition using Task Vectors with Learned Anisotropic Scaling
  3 Jul 2024 1 repository listed Syntology ran 3 of 4 samples · 1 unverified · 4 pointer-only (licence)
  In particular, we show that (1) learned anisotropic scaling allows task vectors to be more disentangled, causing less interference in composition; (2) task vector composition excels with scarce or no labeled data and is…
• No Train but Gain: Language Arithmetic for training-free Language Adapters enhancement
  24 Apr 2024 1 repository listed
  Modular deep learning is the state-of-the-art solution for lifting the curse of multilinguality, preventing the impact of negative interference and enabling cross-lingual performance in Multilingual Pre-trained Language…
• Have You Merged My Model? On The Robustness of Large Language Model IP Protection Methods Against Model Merging
  8 Apr 2024 1 repository listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  Model merging is a promising lightweight model empowerment technique that does not rely on expensive computing devices (e.
• Merging Multi-Task Models via Weight-Ensembling Mixture of Experts
  1 Feb 2024 1 repository listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  A notable challenge is mitigating the interference between parameters of different models, which can substantially deteriorate performance.
• Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion
  11 Dec 2023 1 repository listed Syntology ran 8 of 9 samples · 1 unverified · 9 pointer-only (licence)
  At the upper level, we focus on learning a shared Concrete mask to identify the subspace, while at the inner level, model merging is performed to maximize the performance of the merged model.
• Model Merging by Uncertainty-Based Gradient Matching
  19 Oct 2023 1 repository listed Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
  Models trained on different datasets can be merged by a weighted-averaging of their parameters, but why does it work and when can it fail?
• Parameter Efficient Multi-task Model Fusion with Partial Linearization
  7 Oct 2023 1 repository listed Syntology ran 3 of 4 samples · 1 unverified · 4 pointer-only (licence)
  We demonstrate that our partial linearization technique enables a more effective fusion of multiple tasks into a single model, outperforming standard adapter tuning and task arithmetic alone.
• AdaMerging: Adaptive Model Merging for Multi-Task Learning
  4 Oct 2023 1 repository listed Syntology ran 3 of 3 samples · 0 unverified
  This approach aims to autonomously learn the coefficients for model merging, either in a task-wise or layer-wise manner, without relying on the original training data.
• Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models
  22 May 2023 1 repository listed Syntology ran 1 of 4 samples · 3 unverified
  Task arithmetic has recently emerged as a cost-effective and scalable approach to edit pre-trained models directly in weight space: By adding the fine-tuned weights of different tasks, the model's performance can be…
• An Empirical Study of Multimodal Model Merging
  28 Apr 2023 1 repository listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  In this paper, we expand on this concept to a multimodal setup by merging transformers trained on different modalities.

Syntology lines on 18 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.