Methods › General › Stochastic Optimization › SGD
Stochastic Gradient Descent
SGD
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Stochastic Gradient Descent is an iterative optimization technique that uses minibatches of data to form an expectation of the gradient, rather than the full gradient using all available data. That is for weights w and a loss function L we have:
wₜ₊₁ = wₜ - η∇̂_wL(wₜ)
Where η is a learning rate. SGD reduces redundancy compared to batch gradient descent - which recomputes gradients for similar examples before each parameter update - so it is usually much faster.
(Image Source: here)
Papers archive 2025-07-28
30 shown of 2,021, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Fast Last-Iterate Convergence of SGD in the Smooth Interpolation Regime 15 Jul 2025 · 0 repositories · arXiv:2507.11274
-
A Single Merging Suffices: Recovering Server-based Learning Performance in Decentralized Learning 9 Jul 2025 · 0 repositories · arXiv:2507.06542
-
Tight Generalization Error Bounds for Stochastic Gradient Descent in Non-convex Learning 23 Jun 2025 · 0 repositories · arXiv:2506.18645
-
A Minimalist Optimizer Design for LLM Pretraining 20 Jun 2025 · 1 repository · arXiv:2506.16659Syntology ran 0 of 1 samples · 1 unverified · 1 pointer-only (licence)
-
A Simplified Analysis of SGD for Linear Regression with Weight Averaging 18 Jun 2025 · 0 repositories · arXiv:2506.15535
-
Sharpness-Aware Machine Unlearning 16 Jun 2025 · 0 repositories · arXiv:2506.13715Syntology ran 1 of 1 samples · 0 unverified
-
Is your batch size the problem? Revisiting the Adam-SGD gap in language modeling 14 Jun 2025 · 0 repositories · arXiv:2506.12543
-
Learning single-index models via harmonic decomposition 11 Jun 2025 · 0 repositories · arXiv:2506.09887
-
Improved Scaling Laws in Linear Regression via Data Reuse 10 Jun 2025 · 0 repositories · arXiv:2506.08415
-
Online Learning-guided Learning Rate Adaptation via Gradient Alignment 10 Jun 2025 · 0 repositories · arXiv:2506.08419
-
An Adaptive Method Stabilizing Activations for Enhanced Generalization 10 Jun 2025 · 1 repository · arXiv:2506.08353
-
Leveraging Coordinate Momentum in SignSGD and Muon: Memory-Optimized Zero-Order 4 Jun 2025 · 1 repository · arXiv:2506.04430
-
Classifying Dental Care Providers Through Machine Learning with Features Ranking 4 Jun 2025 · 0 repositories · arXiv:2506.04474
-
Incremental Gradient Descent with Small Epoch Counts is Surprisingly Slow on Ill-Conditioned Problems 4 Jun 2025 · 0 repositories · arXiv:2506.04126
-
Orthogonal Gradient Descent Improves Neural Calibration 4 Jun 2025 · 0 repositories · arXiv:2506.04487
-
Replay Can Provably Increase Forgetting 4 Jun 2025 · 0 repositories · arXiv:2506.04377
-
Asymptotics of SGD in Sequence-Single Index Models and Single-Layer Attention Networks 3 Jun 2025 · 0 repositories · arXiv:2506.02651
-
LightSAM: Parameter-Agnostic Sharpness-Aware Minimization 30 May 2025 · 0 repositories · arXiv:2505.24399
-
SGD as Free Energy Minimization: A Thermodynamic View on Neural Network Training 29 May 2025 · 1 repository · arXiv:2505.23489
-
The Rich and the Simple: On the Implicit Bias of Adam and SGD 29 May 2025 · 0 repositories · arXiv:2505.24022
-
DES-LOC: Desynced Low Communication Adaptive Optimizers for Training Foundation Models 28 May 2025 · 0 repositories · arXiv:2505.22549
-
PADAM: Parallel averaged Adam reduces the error for stochastic optimization in scientific machine learning 28 May 2025 · 0 repositories · arXiv:2505.22085
-
AutoSGD: Automatic Learning Rate Selection for Stochastic Gradient Descent 27 May 2025 · 0 repositories · arXiv:2505.21651
-
Convergence of Clipped-SGD for Convex (L₀,L₁)-Smooth Optimization with Heavy-Tailed Noise 27 May 2025 · 0 repositories · arXiv:2505.20817
-
DeCAF: Decentralized Consensus-And-Factorization for Low-Rank Adaptation of Foundation Models 27 May 2025 · 0 repositories · arXiv:2505.21382
-
Gradient Flow Matching for Learning Update Dynamics in Neural Network Training 26 May 2025 · 0 repositories · arXiv:2505.20221
-
Convergence, Sticking and Escape: Stochastic Dynamics Near Critical Points in SGD 24 May 2025 · 0 repositories · arXiv:2505.18535
-
Accelerating Learned Image Compression Through Modeling Neural Training Dynamics 23 May 2025 · 0 repositories · arXiv:2505.18107
-
Decomposition of Water Demand Patterns Using Skewed Gaussian Distributions for Behavioral Insights and Operational Planning 23 May 2025 · 1 repository · arXiv:2505.18245
-
Emergence of Hebbian Dynamics in Regularized Non-Local Learners 23 May 2025 · 0 repositories · arXiv:2505.18069
Tasks archive 2025-07-28
20 shown of 362 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections