Methods › General › Large Batch Optimization › Nesterov Accelerated Gradient
Nesterov Accelerated Gradient
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Nesterov Accelerated Gradient is a momentum-based SGD optimizer that "looks ahead" to where the parameters will be to calculate the gradient ex post rather than ex ante:
vₜ = γvₜ₋₁ - η∇_θJ(θₜ₋₁+γvₜ₋₁) θₜ = θₜ₋₁ + vₜ γ, η∈ℝ^+
Like SGD with momentum γ is usually set to 0.9. η and γ are usually less than 1.
The intuition is that the standard momentum method first computes the gradient at the current location and then takes a big jump in the direction of the updated accumulated gradient. In contrast Nesterov momentum first makes a big jump in the direction of the previous accumulated gradient and then measures the gradient where it ends up and makes a correction. The idea being that it is better to correct a mistake after you have made it.
Image Source: Geoff Hinton lecture notes
In Syntology name not resolved (unknown_name). The lookup matched no method, paper or venue by this name; that asserts nothing about whether code exists.
Near matches the lookup lists, each a different name: Randomized Subspace Nesterov Accelerated Gradient; Nesterov-accelerated gradient (NAG) enhanced SDE-BNN model; Accelerated Gradient descent algorithm based on Nesterov's smoothing technique; Nesterov-type accelerated scheme; Accelerated Reflected Gradient (ARG) method; Accelerated Optimistic Gradient (AOG) algorithm; Federated Accelerated Stochastic Gradient Descent (FedAc); Accelerated Bregman Gradient Policy Optimization (VR-BGPO).
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 34 · with a code link: 18 · where Syntology ran a sample: 7 (5 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (7 of 34 tagged: 5 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 34 of 34, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
Convergence of Momentum-Based Optimization Algorithms with Time-Varying Parameters 13 Jun 2025 · 0 repositories · arXiv:2506.11904
-
Nesterov Method for Asynchronous Pipeline Parallel Optimization 2 May 2025 · 1 repository · arXiv:2505.01099
-
Advancing RVFL networks: Robust classification with the HawkEye loss function 1 Oct 2024 · 1 repository · arXiv:2410.00510
-
An Accelerated Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness 28 Sep 2024 · 1 repository · arXiv:2409.19212Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples)
-
Optimizing Time Series Forecasting: A Comparative Study of Adam and Nesterov Accelerated Gradient on LSTM and GRU networks Using Stock Market data 28 Sep 2024 · 1 repository · arXiv:2410.01843
-
DenoMamba: A fused state-space model for low-dose CT denoising 19 Sep 2024 · 1 repository · arXiv:2409.13094
-
3D CBCT Challenge 2024: Improved Cone Beam CT Reconstruction using SwinIR-Based Sinogram and Image Enhancement 12 Jun 2024 · 0 repositories · arXiv:2406.08048
-
Momentum-SAM: Sharpness Aware Minimization without Computational Overhead 22 Jan 2024 · 1 repository · arXiv:2401.12033Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
Accelerated gradient methods for nonconvex optimization: Escape trajectories from strict saddle points and convergence to local minima 13 Jul 2023 · 0 repositories · arXiv:2307.07030
-
Riemannian accelerated gradient methods via extrapolation 13 Aug 2022 · 0 repositories · arXiv:2208.06619
-
Last-iterate convergence analysis of stochastic momentum methods for neural networks 30 May 2022 · 0 repositories · arXiv:2205.14811
-
Automated Parking Space Detection Using Convolutional Neural Networks 14 Jun 2021 · 0 repositories · arXiv:2106.07228
-
A Discrete Variational Derivation of Accelerated Methods in Optimization 4 Jun 2021 · 0 repositories · arXiv:2106.02700
-
A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes 12 Feb 2021 · 0 repositories · arXiv:2102.06356
-
Stochastic optimization with momentum: convergence, fluctuations, and traps avoidance 7 Dec 2020 · 0 repositories · arXiv:2012.04002
-
A Dynamical View on Optimization Algorithms of Overparameterized Neural Networks 25 Oct 2020 · 1 repository · arXiv:2010.13165
-
Accelerated Gradient Methods for Sparse Statistical Learning with Nonconvex Penalties 22 Sep 2020 · 1 repository · arXiv:2009.10629
-
Federated Learning with Nesterov Accelerated Gradient 18 Sep 2020 · 0 repositories · arXiv:2009.08716
-
GreedyNAS: Towards Fast One-Shot NAS with Greedy Supernet 25 Mar 2020 · 0 repositories · arXiv:2003.11236
-
Scheduled Restart Momentum for Accelerated Stochastic Gradient Descent 24 Feb 2020 · 1 repository · arXiv:2002.10583Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
Second-order Information in First-order Optimization Methods 20 Dec 2019 · 0 repositories · arXiv:1912.09926
-
LiteSeg: A Novel Lightweight ConvNet for Semantic Segmentation 13 Dec 2019 · 2 repositories · arXiv:1912.06683
-
Best Practices for Convolutional Neural Networks Applied to Object Recognition in Images 29 Oct 2019 · 0 repositories · arXiv:1910.13029
-
Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks 17 Aug 2019 · 3 repositories · arXiv:1908.06281Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
Power Gradient Descent 11 Jun 2019 · 0 repositories · arXiv:1906.04787
-
DANA: Scalable Out-of-the-box Distributed ASGD Without Retuning 1 May 2019 · 0 repositories
-
Bag of Tricks for Image Classification with Convolutional Neural Networks 4 Dec 2018 · 28 repositories · arXiv:1812.01187Syntology community repositories only · 11 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 2 honoured, 0 violated, 7 with no contract checked; 2 where Syntology's instrument failed) · 4 unverified (of 15 harvested samples) · 5 pointer-only (licence)
-
Nesterov Acceleration of Alternating Least Squares for Canonical Tensor Decomposition: Momentum Step Size Selection and Restart Mechanisms 13 Oct 2018 · 1 repository · arXiv:1810.05846
-
A Unified Analysis of AdaGrad with Weighted Aggregation and Momentum Acceleration 10 Aug 2018 · 0 repositories · arXiv:1808.03408
-
Big-Little Net: An Efficient Multi-Scale Feature Representation for Visual and Speech Recognition 10 Jul 2018 · 3 repositories · arXiv:1807.03848
-
Language Modeling with Gated Convolutional Networks 23 Dec 2016 · 11 repositories · arXiv:1612.08083
-
Deep Pyramidal Residual Networks 10 Oct 2016 · 7 repositories · arXiv:1610.02915Syntology community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
Densely Connected Convolutional Networks 25 Aug 2016 · 146 repositories · arXiv:1608.06993Syntology community repositories only · 48 ran (of which 0 constructed an object rather than computing a result; 32 with no instrument failure: 2 honoured, 0 violated, 30 with no contract checked; 16 where Syntology's instrument failed) · 23 unverified (of 71 harvested samples) · 8 pointer-only (licence)
-
Wide Residual Networks 23 May 2016 · 72 repositories · arXiv:1605.07146Syntology official (archive's flag): 3 ran · 66 ran (of which 31 constructed an object rather than computing a result; 47 with no instrument failure: 0 honoured, 0 violated, 47 with no contract checked; 19 where Syntology's instrument failed) · 30 unverified (of 96 harvested samples) · 50 pointer-only (licence)
Tasks archive 2025-07-28
20 shown of 59 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections