Methods › General › Data Parallel Methods › Local SGD
Local SGD
Introduced by Sebastian U. Stich in Local SGD Converges Fast and Communicates Little
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Local SGD is a distributed training technique that runs SGD independently in parallel on different workers and averages the sequences only once in a while.
In Syntology about another paper, arXiv:2011.02828: the paper matched by title held, no code (no_code; paper_only). The lookup matched a paper by its title, arXiv:2011.02828, not a method entry. About another paper: this answer is about arXiv:2011.02828, while the archive credits arXiv:1805.09767, so it does not say whether this method has code.
Origin paper of this answer, arXiv:2011.02828 (its page on Syntology).
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 69 · with a code link: 16 · where Syntology ran a sample: 6 (6 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (6 of 69 tagged: 6 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 69 of 69, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
DES-LOC: Desynced Low Communication Adaptive Optimizers for Training Foundation Models 28 May 2025 · 0 repositories · arXiv:2505.22549
-
Sharp Gaussian approximations for Decentralized Federated Learning 12 May 2025 · 0 repositories · arXiv:2505.08125
-
Streaming Federated Learning with Markovian Data 24 Mar 2025 · 0 repositories · arXiv:2503.18807
-
EDiT: A Local-SGD-Based Efficient Distributed Training Method for Large Language Models 10 Dec 2024 · 2 repositories · arXiv:2412.07210
-
Collaborative and Efficient Personalization with Mixtures of Adaptors 4 Oct 2024 · 0 repositories · arXiv:2410.03497
-
Does Worst-Performing Agent Lead the Pack? Analyzing Agent Dynamics in Unified Distributed SGD 26 Sep 2024 · 0 repositories · arXiv:2409.17499
-
Convergence of Distributed Adaptive Optimization with Local Updates 20 Sep 2024 · 0 repositories · arXiv:2409.13155
-
Exploring Scaling Laws for Local SGD in Large Language Model Training 20 Sep 2024 · 0 repositories · arXiv:2409.13198
-
Communication-Efficient Adaptive Batch Size Strategies for Distributed Local Gradient Methods 20 Jun 2024 · 0 repositories · arXiv:2406.13936
-
Local Methods with Adaptivity via Scaling 2 Jun 2024 · 0 repositories · arXiv:2406.00846
-
The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication 19 May 2024 · 0 repositories · arXiv:2405.11667
-
Communication Efficient and Provable Federated Unlearning 19 Jan 2024 · 1 repository · arXiv:2401.11018
-
Can We Learn Communication-Efficient Optimizers? 2 Dec 2023 · 1 repository · arXiv:2312.02204
-
Asynchronous SGD on Graphs: a Unified Framework for Asynchronous Decentralized and Federated Optimization 1 Nov 2023 · 0 repositories · arXiv:2311.00465
-
A Quadratic Synchronization Rule for Distributed Deep Learning 22 Oct 2023 · 1 repository · arXiv:2310.14423Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples)
-
Asynchronous Federated Learning with Incentive Mechanism Based on Contract Theory 10 Oct 2023 · 0 repositories · arXiv:2310.06448
-
Stability and Generalization for Minibatch SGD and Local SGD 2 Oct 2023 · 0 repositories · arXiv:2310.01139
-
Global Convergence Analysis of Local SGD for Two-layer Neural Network without Overparameterization 21 Sep 2023 · 0 repositories
-
Preconditioned Federated Learning 20 Sep 2023 · 0 repositories · arXiv:2309.11378
-
FedYolo: Augmenting Federated Learning with Pretrained Transformers 10 Jul 2023 · 0 repositories · arXiv:2307.04905
-
Semi-Asynchronous Federated Edge Learning Mechanism via Over-the-air Computation 6 May 2023 · 0 repositories · arXiv:2305.04066
-
Why (and When) does Local SGD Generalize Better than SGD? 2 Mar 2023 · 1 repository · arXiv:2303.01215Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 8 harvested samples)
-
z-SignFedAvg: A Unified Stochastic Sign-based Compression for Federated Learning 6 Feb 2023 · 0 repositories · arXiv:2302.02589
-
When Do Curricula Work in Federated Learning? 24 Dec 2022 · 1 repository · arXiv:2212.12712
-
STSyn: Speeding Up Local SGD with Straggler-Tolerant Synchronization 6 Oct 2022 · 0 repositories · arXiv:2210.03521
-
On the Stability Analysis of Open Federated Learning Systems 25 Sep 2022 · 0 repositories · arXiv:2209.12307
-
FedBR: Improving Federated Learning on Heterogeneous Data via Local Learning Bias Reduction 26 May 2022 · 1 repository · arXiv:2205.13462Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 3 honoured, 0 violated, 1 with no contract checked; 6 where Syntology's instrument failed) · 5 unverified (of 15 harvested samples) · 2 pointer-only (licence)
-
Federated Random Reshuffling with Compression and Variance Reduction 8 May 2022 · 0 repositories · arXiv:2205.03914
-
Federated Stochastic Primal-dual Learning with Differential Privacy 26 Apr 2022 · 0 repositories · arXiv:2204.12284
-
ImageNet Challenging Classification with the Raspberry Pi: An Incremental Local Stochastic Gradient Descent Algorithm 21 Mar 2022 · 0 repositories · arXiv:2203.11853
-
The Role of Local Steps in Local SGD 14 Mar 2022 · 0 repositories · arXiv:2203.06798
-
Faster Convergence of Local SGD for Over-Parameterized Models 30 Jan 2022 · 0 repositories · arXiv:2201.12719
-
Partial Model Averaging in Federated Learning: Performance Guarantees and Benefits 11 Jan 2022 · 0 repositories · arXiv:2201.03789
-
Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective 5 Nov 2021 · 1 repository · arXiv:2111.03741
-
Minibatch vs Local SGD with Shuffling: Tight Convergence Bounds and Beyond 20 Oct 2021 · 0 repositories · arXiv:2110.10342
-
Trade-offs of Local SGD at Scale: An Empirical Study 15 Oct 2021 · 0 repositories · arXiv:2110.08133
-
Hybrid Local SGD for Federated Learning with Heterogeneous Communications 29 Sep 2021 · 0 repositories
-
Statistical Estimation and Inference via Local SGD in Federated Learning 3 Sep 2021 · 0 repositories · arXiv:2109.01326
-
The Minimax Complexity of Distributed Optimization 1 Sep 2021 · 0 repositories · arXiv:2109.00534
-
Byzantine Fault-Tolerance in Federated Local SGD under 2f-Redundancy 26 Aug 2021 · 0 repositories · arXiv:2108.11769
-
FedPAGE: A Fast Local Stochastic Gradient Method for Communication-Efficient Federated Learning 10 Aug 2021 · 0 repositories · arXiv:2108.04755
-
Local SGD Optimizes Overparameterized Neural Networks in Polynomial Time 22 Jul 2021 · 0 repositories · arXiv:2107.10868
-
STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learning 19 Jun 2021 · 0 repositories · arXiv:2106.10435
-
Communication-efficient SGD: From Local SGD to One-Shot Averaging 9 Jun 2021 · 0 repositories · arXiv:2106.04759
-
Accelerating Gossip SGD with Periodic Global Averaging 19 May 2021 · 0 repositories · arXiv:2105.09080
-
Local Stochastic Gradient Descent Ascent: Convergence Analysis and Communication Efficiency 25 Feb 2021 · 0 repositories · arXiv:2102.13152
-
Personalized Federated Learning: A Unified Framework and Universal Optimization Techniques 19 Feb 2021 · 1 repository · arXiv:2102.09743
-
Proactive DP: A Multple Target Optimization Framework for DP-SGD 17 Feb 2021 · 0 repositories · arXiv:2102.09030
-
Proximal and Federated Random Reshuffling 12 Feb 2021 · 1 repository · arXiv:2102.06704Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
Bias-Variance Reduced Local SGD for Less Heterogeneous Federated Learning 5 Feb 2021 · 0 repositories · arXiv:2102.03198
-
Local SGD Meets Asynchrony 1 Jan 2021 · 0 repositories
-
Multi-Level Local SGD: Distributed SGD for Heterogeneous Hierarchical Networks 1 Jan 2021 · 0 repositories
-
Local SGD: Unified Theory and New Efficient Methods 3 Nov 2020 · 0 repositories · arXiv:2011.02828
-
Hogwild! over Distributed Local Data Sets with Linearly Increasing Mini-Batch Sizes 27 Oct 2020 · 0 repositories · arXiv:2010.14763
-
Distributed Saddle-Point Problems: Lower Bounds, Near-Optimal and Robust Algorithms 25 Oct 2020 · 0 repositories · arXiv:2010.13112
-
Over-the-Air Federated Learning from Heterogeneous Data 27 Sep 2020 · 1 repository · arXiv:2009.12787
-
Multi-Level Local SGD for Heterogeneous Hierarchical Networks 27 Jul 2020 · 1 repository · arXiv:2007.13819Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 3 honoured, 0 violated, 2 with no contract checked; 5 where Syntology's instrument failed) · 7 unverified (of 17 harvested samples) · 17 pointer-only (licence)
-
STL-SGD: Speeding Up Local SGD with Stagewise Communication Period 11 Jun 2020 · 0 repositories · arXiv:2006.06377
-
Minibatch vs Local SGD for Heterogeneous Distributed Learning 8 Jun 2020 · 0 repositories · arXiv:2006.04735
-
Local SGD With a Communication Overhead Depending Only on the Number of Workers 3 Jun 2020 · 0 repositories · arXiv:2006.02582
-
SQuARM-SGD: Communication-Efficient Momentum SGD for Decentralized Optimization 13 May 2020 · 0 repositories · arXiv:2005.07041
-
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates 23 Mar 2020 · 0 repositories · arXiv:2003.10422
-
Is Local SGD Better than Minibatch SGD? 18 Feb 2020 · 0 repositories · arXiv:2002.07839
-
Variance Reduced Local SGD with Lower Communication Complexity 30 Dec 2019 · 1 repository · arXiv:1912.12844
-
On the Convergence of Local Descent Methods in Federated Learning 31 Oct 2019 · 0 repositories · arXiv:1910.14425
-
Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronization 30 Oct 2019 · 2 repositories · arXiv:1910.13598Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum 1 Oct 2019 · 2 repositories · arXiv:1910.00643Syntology official: harvested for another paper · 0 ran · 3 unverified (of 3 harvested samples) · 1 pointer-only (licence)
-
Tighter Theory for Local SGD on Identical and Heterogeneous Data 10 Sep 2019 · 0 repositories · arXiv:1909.04746
-
Local SGD Converges Fast and Communicates Little 24 May 2018 · 2 repositories · arXiv:1805.09767
Tasks archive 2025-07-28
20 shown of 23 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections