Methods › General › Stochastic Optimization › Stochastic Weight Averaging
Stochastic Weight Averaging
Introduced by Pavel Izmailov et al. in Averaging Weights Leads to Wider Optima and Better Generalization
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Stochastic Weight Averaging is an optimization procedure that averages multiple points along the trajectory of SGD, with a cyclical or constant learning rate. On the one hand it averages weights, but it also has the property that, with a cyclical or constant learning rate, SGD proposals are approximately sampling from the loss surface of the network, leading to stochastic weights and helping to discover broader optima.
Papers archive 2025-07-28
30 shown of 45, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
An Effective End-to-End Solution for Multimodal Action Recognition 11 Jun 2025 · 0 repositories · arXiv:2506.09345
-
Federated EndoViT: Pretraining Vision Transformers via Federated Learning on Endoscopic Image Collections 23 Apr 2025 · 0 repositories · arXiv:2504.16612
-
Understanding Flatness in Generative Models: Its Role and Benefits 14 Mar 2025 · 0 repositories · arXiv:2503.11078
-
SeWA: Selective Weight Average via Probabilistic Masking 14 Feb 2025 · 0 repositories · arXiv:2502.10119
-
Divide and Conquer: Grounding a Bleeding Areas in Gastrointestinal Image with Two-Stage Model 21 Dec 2024 · 1 repository · arXiv:2412.16723
-
A Unified Analysis for Finite Weight Averaging 20 Nov 2024 · 0 repositories · arXiv:2411.13169
-
Adaptive Stochastic Weight Averaging 27 Jun 2024 · 1 repository · arXiv:2406.19092
-
Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations 28 May 2024 · 4 repositories · arXiv:2405.18392Syntology ran 7 of 13 samples · 6 unverified · 4 pointer-only (licence)
-
Gaussian Stochastic Weight Averaging for Bayesian Low-Rank Adaptation of Large Language Models 6 May 2024 · 1 repository · arXiv:2405.03425Syntology ran 4 of 7 samples · 3 unverified
-
Post-Hoc Reversal: Are We Selecting Models Prematurely? 11 Apr 2024 · 1 repository · arXiv:2404.07815Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
-
Instance Segmentation under Occlusions via Location-aware Copy-Paste Data Augmentation 27 Oct 2023 · 1 repository · arXiv:2310.17949
-
SimBIG: Field-level Simulation-Based Inference of Galaxy Clustering 23 Oct 2023 · 0 repositories · arXiv:2310.15256
-
Weight Averaging Improves Knowledge Distillation under Domain Shift 20 Sep 2023 · 1 repository · arXiv:2309.11446
-
A Novel Training Framework for Physics-informed Neural Networks: Towards Real-time Applications in Ultrafast Ultrasound Blood Flow Imaging 9 Sep 2023 · 0 repositories · arXiv:2309.04755
-
Fine-grained building roof instance segmentation based on domain adapted pretraining and composite dual-backbone 10 Aug 2023 · 0 repositories · arXiv:2308.05358
-
The Split Matters: Flat Minima Methods for Improving the Performance of GNNs 15 Jun 2023 · 1 repository · arXiv:2306.09121
-
Decoupled Training for Long-Tailed Classification With Stochastic Representations 19 Apr 2023 · 0 repositories · arXiv:2304.09426Syntology ran 3 of 14 samples · 11 unverified
-
Training trajectories, mini-batch losses and the curious role of the learning rate 5 Jan 2023 · 0 repositories · arXiv:2301.02312
-
Frequency Regularization for Improving Adversarial Robustness 24 Dec 2022 · 0 repositories · arXiv:2212.12732
-
Improving Generalization of Pre-trained Language Models via Stochastic Weight Averaging 12 Dec 2022 · 0 repositories · arXiv:2212.05956
-
Deep Combinatorial Aggregation 12 Oct 2022 · 1 repository · arXiv:2210.06436Syntology ran 0 of 2 samples · 2 unverified · 2 pointer-only (licence)
-
ARQ-based Average Consensus over Directed Network Topologies with Unreliable Communication Links 29 Sep 2022 · 0 repositories · arXiv:2209.14699
-
Robust Simulation-Based Inference in Cosmology with Bayesian Neural Networks 18 Jul 2022 · 1 repository · arXiv:2207.08435Syntology ran 2 of 5 samples · 3 unverified · 5 pointer-only (licence)
-
Trainable Weight Averaging: A General Approach for Subspace Training 26 May 2022 · 1 repository · arXiv:2205.13104Syntology ran 5 of 8 samples · 3 unverified
-
UNBUS: Uncertainty-aware Deep Botnet Detection System in Presence of Perturbed Samples 18 Apr 2022 · 0 repositories · arXiv:2204.09502
-
Improving Generalization in Federated Learning by Seeking Flat Minima 22 Mar 2022 · 1 repository · arXiv:2203.11834Syntology ran 1 of 1 samples · 0 unverified
-
PFGE: Parsimonious Fast Geometric Ensembling of DNNs 14 Feb 2022 · 1 repository · arXiv:2202.06658
-
When Do Flat Minima Optimizers Work? 1 Feb 2022 · 1 repository · arXiv:2202.00661
-
SpectraNet: Learned Recognition of Artificial Satellites From High Contrast Spectroscopic Imagery 10 Jan 2022 · 0 repositories · arXiv:2201.03614
-
Stochastic Weight Averaging Revisited 3 Jan 2022 · 1 repository · arXiv:2201.00519
Tasks archive 2025-07-28
20 shown of 69 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections