Linear Mode Connectivity

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

No dataset record in the archive lists this task.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

14 shown of 14 papers with code (35 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Git Re-Basin: Merging Models modulo Permutation Symmetries
  11 Sep 2022 3 repositories listed Syntology ran 6 of 6 samples · 0 unverified
  The success of deep learning is due in large part to our ability to solve certain massive non-convex optimization problems with relative ease.
• Federated Learning over Connected Modes
  5 Mar 2024 2 repositories listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  Statistical heterogeneity in federated learning poses two major challenges: slow global training due to conflicting gradient signals, and the need of personalization for local distributions.
• Vanishing Feature: Diagnosing Model Merging and Beyond
  5 Feb 2024 2 repositories listed Syntology ran 2 of 3 samples · 1 unverified · 3 pointer-only (licence)
  Model merging offers an efficient way to combine pre-trained neural networks but often suffers from inconsistent performance, especially when merging models with different initializations.
• Proving Linear Mode Connectivity of Neural Networks via Optimal Transport
  29 Oct 2023 2 repositories listed
  The energy landscape of high-dimensional non-convex optimization problems is crucial to understanding the effectiveness of modern deep neural network architectures.
• Linear Mode Connectivity and the Lottery Ticket Hypothesis
  11 Dec 2019 2 repositories listed
  We study whether a neural network optimizes to the same, linearly connected minimum under different samples of SGD noise (e.
• Deep Learning Through A Telescoping Lens: A Simple Model Provides Empirical Insights On Grokking, Gradient Boosting & Beyond
  31 Oct 2024 1 repository listed
  Deep learning sometimes appears to work in unexpected ways.
• Approaching Deep Learning through the Spectral Dynamics of Weights
  21 Aug 2024 1 repository listed Syntology ran 9 of 14 samples · 5 unverified
  We propose an empirical approach centered on the spectral dynamics of weights -- the behavior of singular values and vectors during optimization -- to unify and clarify several phenomena in deep learning.
• The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof
  30 May 2024 1 repository listed Syntology ran 1 of 1 samples · 0 unverified
  Our experiments reveal several interesting observations on the empirical impact of parameter symmetries; for instance, we observe linear mode connectivity between our networks without alignment of weight spaces, and we…
• Layer-wise Linear Mode Connectivity
  13 Jul 2023 1 repository listed Syntology ran 13 of 18 samples · 5 unverified · 18 pointer-only (licence)
  Averaging neural network parameters is an intuitive method for fusing the knowledge of two independent models.
• Lottery Tickets in Evolutionary Optimization: On Sparse Backpropagation-Free Trainability
  31 May 2023 1 repository listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  Is the lottery ticket phenomenon an idiosyncrasy of gradient-based training or does it generalize to evolutionary optimization?
• Re-basin via implicit Sinkhorn differentiation
  22 Dec 2022 1 repository listed Syntology ran 1 of 1 samples · 0 unverified
  The recent emergence of new algorithms for permuting models into functionally equivalent regions of the solution space has shed some light on the complexity of error surfaces, and some promising properties like mode…
• Wasserstein Barycenter-based Model Fusion and Linear Mode Connectivity of Neural Networks
  13 Oct 2022 1 repository listed
  In our framework, the fusion occurs in a layer-wise manner and builds on an interpretation of a node in a network as a function of the layer preceding it.
• The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks
  12 Oct 2021 1 repository listed
  In this paper, we conjecture that if the permutation invariance of neural networks is taken into account, SGD solutions will likely have no barrier in the linear interpolation between them.
• Linear Mode Connectivity in Multitask and Continual Learning
  9 Oct 2020 1 repository listed
  Continual (sequential) training and multitask (simultaneous) training are often attempting to solve the same overall objective: to find a solution that performs well on all considered tasks.

Syntology lines on 8 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.