Browse State-of-the-Art › Linear Mode Connectivity
Linear Mode Connectivity
14 papers with code · 0 benchmarks · 0 datasets archive 2025-07-28
Linear Mode Connectivity refers to the relationship between input and output variables in a linear regression model. In a linear regression model, input variables are combined with weights to predict output variables. Understanding the linear model connectivity can help interpret model results and identify which input variables are most important for predicting output variables.
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
No benchmark for this task in the archive.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
No dataset record in the archive lists this task.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 35 · with a code link: 14 · where Syntology ran a sample: 9 (8 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (9 of 35 tagged: 8 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 35 of 35, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
11 Sep 2022 3 repositories listed Syntology official (archive's flag): 5 ran · 6 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 5 where Syntology's instrument failed) · 0 unverified (of 6 harvested samples) · 1 pointer-only (licence)The success of deep learning is due in large part to our ability to solve certain massive non-convex optimization problems with relative ease.
-
5 Mar 2024 2 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)Statistical heterogeneity in federated learning poses two major challenges: slow global training due to conflicting gradient signals, and the need of personalization for local distributions.
-
5 Feb 2024 2 repositories listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples) · 3 pointer-only (licence)Model merging offers an efficient way to combine pre-trained neural networks but often suffers from inconsistent performance, especially when merging models with different initializations.
-
29 Oct 2023 2 repositories listedThe energy landscape of high-dimensional non-convex optimization problems is crucial to understanding the effectiveness of modern deep neural network architectures.
-
11 Dec 2019 2 repositories listedWe study whether a neural network optimizes to the same, linearly connected minimum under different samples of SGD noise (e. g., random data order and augmentation).
-
31 Oct 2024 1 repository listedDeep learning sometimes appears to work in unexpected ways.
-
21 Aug 2024 1 repository listed Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 1 honoured, 1 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 5 unverified (of 14 harvested samples)We propose an empirical approach centered on the spectral dynamics of weights -- the behavior of singular values and vectors during optimization -- to unify and clarify several phenomena in deep learning.
-
30 May 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)Our experiments reveal several interesting observations on the empirical impact of parameter symmetries; for instance, we observe linear mode connectivity between our networks without alignment of weight spaces, and we…
-
13 Jul 2023 1 repository listed Syntology official (archive's flag): 13 ran · 13 ran (of which 0 constructed an object rather than computing a result; 13 with no instrument failure: 0 honoured, 0 violated, 13 with no contract checked; 0 where Syntology's instrument failed) · 5 unverified (of 18 harvested samples) · 18 pointer-only (licence)Averaging neural network parameters is an intuitive method for fusing the knowledge of two independent models.
-
31 May 2023 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)Is the lottery ticket phenomenon an idiosyncrasy of gradient-based training or does it generalize to evolutionary optimization?
-
22 Dec 2022 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)The recent emergence of new algorithms for permuting models into functionally equivalent regions of the solution space has shed some light on the complexity of error surfaces, and some promising properties like mode…
-
13 Oct 2022 1 repository listedIn our framework, the fusion occurs in a layer-wise manner and builds on an interpretation of a node in a network as a function of the layer preceding it.
-
12 Oct 2021 1 repository listedIn this paper, we conjecture that if the permutation invariance of neural networks is taken into account, SGD solutions will likely have no barrier in the linear interpolation between them.
-
9 Oct 2020 1 repository listedContinual (sequential) training and multitask (simultaneous) training are often attempting to solve the same overall objective: to find a solution that performs well on all considered tasks.
-
Understanding Mode Connectivity via Parameter Space Symmetry29 May 2025 0 repositories listedNeural network minima are often connected by curves along which train and test loss remain nearly constant, a phenomenon known as mode connectivity.
-
CodeMerge: Codebook-Guided Model Merging for Robust Test-Time Adaptation in Autonomous Driving22 May 2025 0 repositories listedMaintaining robust 3D perception under dynamic and unpredictable test-time conditions remains a critical challenge for autonomous driving systems.
-
Model Assembly Learning with Heterogeneous Layer Weight Merging27 Mar 2025 0 repositories listedModel merging acquires general capabilities without extra data or training by combining multiple models' parameters.
-
Finding Stable Subnetworks at Initialization with Dataset Distillation23 Mar 2025 0 repositories listedRecent works have shown that Dataset Distillation, the process for summarizing the training data, can be leveraged to accelerate the training of deep learning models.
-
Analyzing the Role of Permutation Invariance in Linear Mode Connectivity8 Mar 2025 0 repositories listedIt was empirically observed in Entezari et al. (2021) that when accounting for the permutation invariance of neural networks, there is likely no loss barrier along the linear interpolation between two SGD solutions -- a…
-
24 Feb 2025 0 repositories listed Syntology 6 ran (of which 3 constructed an object rather than computing a result; 6 with no instrument failure: 3 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples) · 7 pointer-only (licence)Recent studies have shown that reducing symmetries in neural networks enhances linear mode connectivity between networks without requiring parameter space alignment, leading to improved performance in linearly…
-
CopRA: A Progressive LoRA Training Strategy30 Oct 2024 0 repositories listedLow-Rank Adaptation (LoRA) is a parameter-efficient technique for rapidly fine-tuning foundation models.
-
The Non-Local Model Merging Problem: Permutation Symmetries and Variance Collapse16 Oct 2024 0 repositories listedModel merging aims to efficiently combine the weights of multiple expert models, each trained on a specific task, into a single multi-task model, with strong performance across all tasks.
-
Landscaping Linear Mode Connectivity24 Jun 2024 0 repositories listedThe presence of linear paths in parameter space between two different network solutions in certain cases, i.e., linear mode connectivity (LMC), has garnered interest from both theoretical and practical fronts.
-
Unveiling the Dynamics of Information Interplay in Supervised Learning6 Jun 2024 0 repositories listedIn this paper, we use matrix information theory as an analytical tool to analyze the dynamics of the information interplay between data representations and classification head vectors in the supervised learning process.
-
Linear Mode Connectivity in Differentiable Tree Ensembles23 May 2024 0 repositories listedLinear Mode Connectivity (LMC) refers to the phenomenon that performance remains consistent for linearly interpolated models in the parameter space.
-
Improving Group Connectivity for Generalization of Federated Deep Learning29 Feb 2024 0 repositories listedFederated learning (FL) involves multiple heterogeneous clients collaboratively training a global model via iterative local updates and model fusion.
-
Analysis of Linear Mode Connectivity via Permutation-Based Weight Matching6 Feb 2024 0 repositories listedRecently, Ainsworth et al. showed that using weight matching (WM) to minimize the L₂ distance in a permutation search of model parameters effectively identifies permutations that satisfy linear mode connectivity (LMC)…
-
Training-time Neuron Alignment through Permutation Subspace for Improving Linear Mode Connectivity and Model Fusion2 Feb 2024 0 repositories listedIn deep learning, stochastic gradient descent often yields functionally similar yet widely scattered solutions in the weight space even under the same initialization, causing barriers in the Linear Mode Connectivity…
-
Disentangling Linear Mode-Connectivity15 Dec 2023 0 repositories listedLinear mode-connectivity (LMC) (or lack thereof) is one of the intriguing characteristics of neural network loss landscapes.
-
Linear Mode Connectivity in Sparse Neural Networks28 Oct 2023 0 repositories listedWith the rise in interest of sparse neural networks, we study how neural network pruning with synthetic data leads to sparse networks with unique training properties.
-
Mode Connectivity and Data Heterogeneity of Federated Learning29 Sep 2023 0 repositories listed
-
High-dimensional manifold of solutions in neural networks: insights from statistical physics17 Sep 2023 0 repositories listed
-
Mode Combinability: Exploring Convex Combinations of Permutation Aligned Models22 Aug 2023 0 repositories listed
-
On Privileged and Convergent Bases in Neural Network Representations24 Jul 2023 0 repositories listed
-
Towards Understanding Iterative Magnitude Pruning: Why Lottery Tickets Win13 Jun 2021 0 repositories listed
Syntology lines on 9 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections