Methods › General › Stochastic Optimization › NT-ASGD

Non-monotonically Triggered ASGD

NT-ASGD

6 papers tagged archive 2025-07-28

Introduced by Stephen Merity et al. in Regularizing and Optimizing LSTM Language Models

archive 2025-07-28 Description, source and code snippet are the archive's method entry.

NT-ASGD, or Non-monotonically Triggered ASGD, is an averaged stochastic gradient descent technique.

In regular ASGD, we take steps identical to regular SGD but instead of returning the last iterate as the solution, we return 1/(K-T+1)∑ᵀ_(i=T)wᵢ, where K is the total number of iterations and T < K is a user-specified averaging trigger.

NT-ASGD has a non-monotonic criterion that conservatively triggers the averaging when the validation metric fails to improve for multiple cycles. Given that the choice of triggering is irreversible, this conservatism ensures that the randomness of training does not play a major role in the decision.

Paper Source See Code · salesforce/awd-lstm-lm

Papers archive 2025-07-28

6 shown of 6, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.

Probing for Referential Information in Language Models
1 Jul 2020 · 0 repositories
MaxUp: A Simple Way to Improve Generalization of Neural Network Training
20 Feb 2020 · 1 repository · arXiv:2002.09024

Syntology ran 0 of 1 samples · 1 unverified
DeFINE: DEep Factorized INput Token Embeddings for Neural Sequence Modeling
27 Nov 2019 · 1 repository · arXiv:1911.12385
A Subword Level Language Model for Bangla Language
15 Nov 2019 · 0 repositories · arXiv:1911.07613
Language Informed Modeling of Code-Switched Text
1 Jul 2018 · 0 repositories
Regularizing and Optimizing LSTM Language Models
7 Aug 2017 · 45 repositories · arXiv:1708.02182

Syntology ran 6 of 7 samples · 1 unverified · 7 pointer-only (licence)

Tasks archive 2025-07-28

15 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.

Task	Papers
Language Modeling	4
Language Modelling	4
Translation	3
Image Classification	2
Machine Translation	2
Articles	1
Few-Shot Image Classification	1
General Classification	1
Sentence	1
Speech Recognition	1
Text Classification	1
Word Embeddings	1
image-classification	1
model	1
text-classification	1

Usage over time archive 2025-07-28

Papers per year the archive tags with this method, by the paper's archive date (6 dated). Bars are counts, not a trend claim.

Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).

Categories archive 2025-07-28

Stochastic Optimization

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections