Distributed Optimization

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

No dataset record in the archive lists this task.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Parent tasks archive 2025-07-28

Most implemented papers archive 2025-07-28

30 shown of 86 papers with code (536 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Federated Optimization in Heterogeneous Networks
  14 Dec 2018 22 repositories listed Syntology ran 9 of 26 samples · 17 unverified · 9 pointer-only (licence)
  Theoretically, we provide convergence guarantees for our framework when learning over data from non-identical distributions (statistical heterogeneity), and while adhering to device-level systems constraints by allowing…
• SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
  14 Oct 2019 8 repositories listed Syntology ran 0 of 4 samples · 4 unverified
  We obtain tight convergence rates for FedAvg and prove that it suffers from `client-drift' when the data is heterogeneous (non-iid), resulting in unstable and slow convergence.
• Power Bundle Adjustment for Large-Scale 3D Reconstruction
  27 Apr 2022 3 repositories listed
  We demonstrate that employing the proposed Power Bundle Adjustment as a sub-problem solver significantly improves speed and accuracy of the distributed optimization.
• Secure Distributed Training at Scale
  21 Jun 2021 3 repositories listed Syntology ran 0 of 9 samples · 9 unverified
  Training such models requires a lot of computational resources (e.
• ZOOpt: Toolbox for Derivative-Free Optimization
  31 Dec 2017 3 repositories listed
  Recent advances in derivative-free optimization allow efficient approximation of the global-optimal solutions of sophisticated functions, such as functions with many local optima, non-differentiable and non-continuous…
• Distributed Adversarial Training to Robustify Deep Neural Networks at Scale
  13 Jun 2022 2 repositories listed
  Spurred by that, we propose distributed adversarial training (DAT), a large-batch adversarial training framework implemented over multiple machines.
• DeepLM: Large-Scale Nonlinear Least Squares on Deep Learning Frameworks Using Stochastic Domain Decomposition
  19 Jun 2021 2 repositories listed
  We propose a novel approach for large-scale nonlinear least squares problems based on deep learning frameworks.
• Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices
  4 Mar 2021 2 repositories listed Syntology ran 1 of 1 samples · 0 unverified
  Training deep neural networks on large datasets can often be accelerated by using multiple compute nodes.
• Training Large Neural Networks with Constant Memory using a New Execution Algorithm
  13 Feb 2020 2 repositories listed Syntology ran 0 of 3 samples · 3 unverified
  By running the optimizer in the host EPS, we show a new form of mixed precision for faster throughput and convergence.
• FedDANE: A Federated Newton-Type Method
  7 Jan 2020 2 repositories listed Syntology ran 1 of 1 samples · 0 unverified
  Federated learning aims to jointly learn statistical models over massively distributed remote devices.
• Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronization
  30 Oct 2019 2 repositories listed Syntology ran 0 of 1 samples · 1 unverified
  Specifically, we show that for loss functions that satisfy the Polyak-{\L}ojasiewicz condition, O((pT)^(1/3)) rounds of communication suffice to achieve a linear speed up, that is, an error of O(1/pT), where T is the…
• SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum
  1 Oct 2019 2 repositories listed Syntology ran 0 of 3 samples · 3 unverified
  We provide theoretical convergence guarantees showing that SlowMo converges to a stationary point of smooth non-convex losses.
• Federated Learning: Challenges, Methods, and Future Directions
  21 Aug 2019 2 repositories listed Syntology ran 5 of 11 samples · 6 unverified
  Federated learning involves training statistical models over remote devices or siloed data centers, such as mobile phones or hospitals, while keeping data localized.
• Robust Learning from Untrusted Sources
  29 Jan 2019 2 repositories listed
  Modern machine learning methods often require more data for training than a single expert can provide.
• CoCoA: A General Framework for Communication-Efficient Distributed Optimization
  7 Nov 2016 2 repositories listed
  The scale of modern datasets necessitates the development of efficient distributed optimization methods for machine learning.
• L1-Regularized Distributed Optimization: A Communication-Efficient Primal-Dual Framework
  13 Dec 2015 2 repositories listed
  Despite the importance of sparsity in many large-scale applications, there are few methods for distributed optimization of sparsity-inducing objectives.
• DPLib: A Standard Benchmark Library for Distributed Power System Analysis and Optimization
  25 Jun 2025 1 repository listed
  Numerical results validate the generated test cases, establishing DPLib as a foundation for reproducible distributed power system research.
• Opportunistic Routing in Wireless Communications via Learnable State-Augmented Policies
  5 Mar 2025 1 repository listed
  This paper addresses the challenge of packet-based information routing in large-scale wireless communication networks.
• FedCFA: Alleviating Simpson's Paradox in Model Aggregation with Counterfactual Federated Learning
  25 Dec 2024 1 repository listed
  Thus, we propose FedCFA, a novel FL framework employing counterfactual learning to generate counterfactual samples by replacing local data critical factors with global average data, aligning local data distributions…
• Optimization Algorithm Design via Electric Circuits
  4 Nov 2024 1 repository listed Syntology ran 0 of 8 samples · 8 unverified
  We present a novel methodology for convex optimization algorithm design using ideas from electric RLC circuits.
• GNN-Empowered Effective Partial Observation MARL Method for AoI Management in Multi-UAV Network
  18 Aug 2024 1 repository listed
  Due to the discretization and temporal features of AoI indicators, the Qedgix framework employs QMIX to optimize distributed partially observable Markov decision processes (Dec-POMDP) based on centralized training and…
• Communication- and Computation-Efficient Distributed Submodular Optimization in Robot Mesh Networks
  15 Jul 2024 1 repository listed
  We demonstrate RAG's performance in simulated scenarios of area detection with up to 45 robots, simulating realistic robot-to-robot (r2r) communication speeds such as the 0.
• ACCO: Accumulate While You Communicate for Communication-Overlapped Sharded LLM Training
  3 Jun 2024 1 repository listed
  Training LLMs relies on distributed implementations using multiple GPUs to compute gradients in parallel with sharded optimizers.
• MicroAdam: Accurate Adaptive Optimization with Low Space Overhead and Provable Convergence
  24 May 2024 1 repository listed Syntology ran 7 of 10 samples · 3 unverified
  We propose a new variant of the Adam optimizer called MicroAdam that specifically minimizes memory overheads, while maintaining theoretical convergence guarantees.
• PIM-Opt: Demystifying Distributed Optimization Algorithms on a Real-World Processing-In-Memory System
  10 Apr 2024 1 repository listed
  Our results demonstrate three major findings: 1) The UPMEM PIM system can be a viable alternative to state-of-the-art CPUs and GPUs for many memory-bound ML training workloads, especially when operations and datatypes…
• FairSync: Ensuring Amortized Group Exposure in Distributed Recommendation Retrieval
  16 Feb 2024 1 repository listed
  Specifically, FairSync resolves the issue by moving it to the dual space, where a central node aggregates historical fairness data into a vector and distributes it to all servers.
• Distributed Markov Chain Monte Carlo Sampling based on the Alternating Direction Method of Multipliers
  29 Jan 2024 1 repository listed
  Many machine learning applications require operating on a spatially distributed dataset.
• Asynchronous Local-SGD Training for Language Modeling
  17 Jan 2024 1 repository listed
  Local stochastic gradient descent (Local-SGD), also referred to as federated averaging, is an approach to distributed optimization where each device performs more than one SGD update per communication.
• Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Rates
  15 Oct 2023 1 repository listed Syntology ran 0 of 1 samples · 1 unverified · 1 pointer-only (licence)
  Byzantine robustness is an essential feature of algorithms for certain distributed optimization problems, typically encountered in collaborative/federated learning.
• Transmission Investment Coordination using MILP Lagrange Dual Decomposition and Auxiliary Problem Principle
  10 Oct 2023 1 repository listed
  This incentive mechanism can be viewed as a set of rules of the transmission expansion investment coordination game, set by the social planner TPC, such that, even if the individual TPs act selfishly, it will still lead…

Syntology lines on 12 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.