Methods › General › Stochastic Optimization › Adam
Adam
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Adam is an adaptive learning rate optimization algorithm that utilises both momentum and scaling, combining the benefits of RMSProp and SGD w/th Momentum. The optimizer is designed to be appropriate for non-stationary objectives and problems with very noisy and/or sparse gradients.
The weight updates are performed as:
wₜ = wₜ₋₁ - ηm̂ₜ/(√(v̂ₜ) + ϵ)
with
m̂ₜ = mₜ/(1-βᵗ₁)
v̂ₜ = vₜ/(1-βᵗ₂)
mₜ = β₁mₜ₋₁ + (1-β₁)gₜ
vₜ = β₂vₜ₋₁ + (1-β₂)gₜ²
η is the step size/learning rate, around 1e-3 in the original paper. ϵ is a small number, typically 1e-8 or 1e-10, to prevent dividing by zero. β₁ and β₂ are forgetting parameters, with typical values 0.9 and 0.999, respectively.
In Syntology repository known, no verified implementation (declared_code; repositories: 5 listed of 87 in the graph, strongest link: mentioned on GitHub). The archive credits no paper with an arXiv id for this method, so whether this answer (about arXiv:1412.6980) is about this method is not checked.
Code Syntology holds under other names, as the lookup lists them (code_nearby); not a complete list, and none of them is this method.
- SUPER-ADAM · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='SUPER-ADAM') - Cayley ADAM · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='Cayley ADAM') - Adam-like extension of SAM · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='Adam-like extension of SAM') - Adaptation-Aware kernel Modulation (AdAM) · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='Adaptation-Aware kernel Modulation (AdAM)')
Origin paper of this answer, arXiv:1412.6980 (its page on this site · its page on Syntology): Syntology's run record for its harvested code, 27 ran (of which 7 constructed an object rather than computing a result; 23 with no instrument failure: 4 honoured, 2 violated, 17 with no contract checked; 4 where Syntology's instrument failed) · 10 unverified (of 37 harvested samples) · 8 pointer-only (licence). The instrument figure counts failures of Syntology's instrument, not of the code. Ran means executed on a synthesized input, not that the code is correct.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
What each level means, quoted from Syntology's server card: V0 “imports”; V1 “executes against a synthesized fixture and returns a value of the recorded type”; V2 “two implementations GENERATED from the paper's signature and spec agree numerically (1e-6) on fixture inputs” “neither of the two ever read the authors' own code, so V2 measures how unambiguously the paper specifies the method, not fidelity to the authors' implementation (measured 2026-09-11, commit 75eaa77: cross_checked_by is a second LLM on 2,628 of 2,831 served samples)”; V3 “paper-derived property tests pass on both of those same two generations.” The card adds: “No rung on this ladder compares the served code against the authors' implementation, so treat every level as evidence about runnability and internal agreement, not about matching the paper's results.” A level quoted in two parts is one definition the card writes as two clauses joined by a double hyphen, each clause quoted on its own (how these levels are quoted).
Papers archive 2025-07-28
archive papers tagged: 24,390 · with a code link: 10,944 · where Syntology ran a sample: 3,424 (2,801 with a run with no instrument failure, 623 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (3,424 of 24,390 tagged: 2,801 with a run with no instrument failure, 623 where every run was a failure of Syntology's instrument)
Page 1 of 244: papers 1 to 100 of 24,390, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
Constructing and Evaluating Declarative RAG Pipelines in PyTerrier 12 Jun 2025 · 1 repository · arXiv:2506.10802
-
PyLO: Towards Accessible Learned Optimizers in PyTorch 12 Jun 2025 · 1 repository · arXiv:2506.10315
-
Towards Robust Multimodal Emotion Recognition under Missing Modalities and Distribution Shifts 12 Jun 2025 · 1 repository · arXiv:2506.10452
-
A Novel Lightweight Transformer with Edge-Aware Fusion for Remote Sensing Image Captioning 11 Jun 2025 · 0 repositories · arXiv:2506.09429
-
Auto-Compressing Networks 11 Jun 2025 · 0 repositories · arXiv:2506.09714
-
Learning Efficient and Generalizable Graph Retriever for Knowledge-Graph Question Answering 11 Jun 2025 · 1 repository · arXiv:2506.09645
-
SparseSSM: Efficient Selective Structured State Space Models Can Be Pruned in One-Shot 11 Jun 2025 · 0 repositories · arXiv:2506.09613
-
Online Learning-guided Learning Rate Adaptation via Gradient Alignment 10 Jun 2025 · 0 repositories · arXiv:2506.08419
-
ADAM: Autonomous Discovery and Annotation Model using LLMs for Context-Aware Annotations 10 Jun 2025 · 0 repositories · arXiv:2506.08968
-
An Adaptive Method Stabilizing Activations for Enhanced Generalization 10 Jun 2025 · 1 repository · arXiv:2506.08353
-
FZOO: Fast Zeroth-Order Optimizer for Fine-Tuning Large Language Models towards Adam-Scale Speed 10 Jun 2025 · 0 repositories · arXiv:2506.09034
-
Hierarchical Neural Collapse Detection Transformer for Class Incremental Object Detection 10 Jun 2025 · 0 repositories · arXiv:2506.08562
-
Hyperspectral Image Classification via Transformer-based Spectral-Spatial Attention Decoupling and Adaptive Gating 10 Jun 2025 · 1 repository · arXiv:2506.08324
-
MedMoE: Modality-Specialized Mixture of Experts for Medical Vision-Language Understanding 10 Jun 2025 · 0 repositories · arXiv:2506.08356
-
MetaTT: A Global Tensor-Train Adapter for Parameter-Efficient Fine-Tuning 10 Jun 2025 · 0 repositories · arXiv:2506.09105
-
PatchGuard: Adversarially Robust Anomaly Detection and Localization through Vision Transformers and Pseudo Anomalies 10 Jun 2025 · 2 repositories · arXiv:2506.09237
-
Robust Visual Localization via Semantic-Guided Multi-Scale Transformer 10 Jun 2025 · 0 repositories · arXiv:2506.08526
-
TACTIC: Translation Agents with Cognitive-Theoretic Interactive Collaboration 10 Jun 2025 · 1 repository · arXiv:2506.08403
-
4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos 9 Jun 2025 · 0 repositories · arXiv:2506.08015
-
Lightweight Sequential Transformers for Blood Glucose Level Prediction in Type-1 Diabetes 9 Jun 2025 · 0 repositories · arXiv:2506.07864
-
LlamaRec-LKG-RAG: A Single-Pass, Learnable Knowledge Graph-RAG Framework for LLM-Based Ranking 9 Jun 2025 · 1 repository · arXiv:2506.07449
-
LLM-driven Indoor Scene Layout Generation via Scaled Human-aligned Data Synthesis and Multi-Stage Preference Optimization 9 Jun 2025 · 0 repositories · arXiv:2506.07570
-
MADFormer: Mixed Autoregressive and Diffusion Transformers for Continuous Image Generation 9 Jun 2025 · 0 repositories · arXiv:2506.07999
-
Quantum Graph Transformer for NLP Sentiment Classification 9 Jun 2025 · 0 repositories · arXiv:2506.07937
-
SceneRAG: Scene-level Retrieval-Augmented Generation for Video Understanding 9 Jun 2025 · 0 repositories · arXiv:2506.07600
-
Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models 8 Jun 2025 · 0 repositories · arXiv:2506.07121
-
Adam assisted Fully informed Particle Swarm Optimzation ( Adam-FIPSO ) based Parameter Prediction for the Quantum Approximate Optimization Algorithm (QAOA) 7 Jun 2025 · 0 repositories · arXiv:2506.06790
-
Breaking Data Silos: Towards Open and Scalable Mobility Foundation Models via Generative Continual Learning 7 Jun 2025 · 0 repositories · arXiv:2506.06694
-
Can In-Context Reinforcement Learning Recover From Reward Poisoning Attacks? 7 Jun 2025 · 0 repositories · arXiv:2506.06891
-
BEAST: Efficient Tokenization of B-Splines Encoded Action Sequences for Imitation Learning 6 Jun 2025 · 0 repositories · arXiv:2506.06072
-
Direct Behavior Optimization: Unlocking the Potential of Lightweight LLMs 6 Jun 2025 · 0 repositories · arXiv:2506.06401
-
Gradual Transition from Bellman Optimality Operator to Bellman Operator in Online Reinforcement Learning 6 Jun 2025 · 1 repository · arXiv:2506.05968Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 4 harvested samples)
-
When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation 6 Jun 2025 · 1 repository · arXiv:2506.05690Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples)
-
Adaptive Preconditioners Trigger Loss Spikes in Adam 5 Jun 2025 · 0 repositories · arXiv:2506.04805
-
Grapheme-Coherent Phonemic and Prosodic Annotation of Speech by Implicit and Explicit Grapheme Conditioning 5 Jun 2025 · 0 repositories · arXiv:2506.04527
-
Intentionally Unintentional: GenAI Exceptionalism and the First Amendment 5 Jun 2025 · 0 repositories · arXiv:2506.05211
-
On Automating Security Policies with Contemporary LLMs 5 Jun 2025 · 0 repositories · arXiv:2506.04838
-
The NTNU System at the S&I Challenge 2025 SLA Open Track 5 Jun 2025 · 0 repositories · arXiv:2506.05121
-
Leveraging Coordinate Momentum in SignSGD and Muon: Memory-Optimized Zero-Order 4 Jun 2025 · 1 repository · arXiv:2506.04430
-
Facts are Harder Than Opinions -- A Multilingual, Comparative Analysis of LLM-Based Fact-Checking Reliability 4 Jun 2025 · 0 repositories · arXiv:2506.03655
-
Magic Mushroom: A Customizable Benchmark for Fine-grained Analysis of Retrieval Noise Erosion in RAG Systems 4 Jun 2025 · 0 repositories · arXiv:2506.03901
-
Privacy and Security Threat for OpenAI GPTs 4 Jun 2025 · 0 repositories · arXiv:2506.04036
-
TracLLM: A Generic Framework for Attributing Long Context LLMs 4 Jun 2025 · 1 repository · arXiv:2506.04202
-
A Novel Deep Reinforcement Learning Method for Computation Offloading in Multi-User Mobile Edge Computing with Decentralization 3 Jun 2025 · 0 repositories · arXiv:2506.02458
-
An Exploratory Framework for Future SETI Applications: Detecting Generative Reactivity via Language Models 3 Jun 2025 · 0 repositories · arXiv:2506.02730
-
Enhancing Automatic PT Tagging for MEDLINE Citations Using Transformer-Based Models 3 Jun 2025 · 0 repositories · arXiv:2506.03321
-
Rethinking the effects of data contamination in Code Intelligence 3 Jun 2025 · 0 repositories · arXiv:2506.02791
-
Unit Commitment with Cost-Oriented Temporal Resolution 3 Jun 2025 · 0 repositories · arXiv:2506.02707
-
Automatic Stage Lighting Control: Is it a Rule-Driven Process or Generative Task? 2 Jun 2025 · 1 repository · arXiv:2506.01482
-
LLM in the Loop: Creating the PARADEHATE Dataset for Hate Speech Detoxification 2 Jun 2025 · 0 repositories · arXiv:2506.01484
-
LLMs as World Models: Data-Driven and Human-Centered Pre-Event Simulation for Disaster Impact Assessment 2 Jun 2025 · 0 repositories · arXiv:2506.06355
-
Retrieval-Augmented Generation of Ontologies from Relational Databases 2 Jun 2025 · 0 repositories · arXiv:2506.01232
-
CLAP-ART: Automated Audio Captioning with Semantic-rich Audio Representation Tokenizer 1 Jun 2025 · 0 repositories · arXiv:2506.00800
-
How Neural Networks Organize Concepts: Introducing Concept Trajectory Analysis for Deep Learning Interpretability 1 Jun 2025 · 1 repository
-
RARE: Retrieval-Aware Robustness Evaluation for Retrieval-Augmented Generation Systems 1 Jun 2025 · 1 repository · arXiv:2506.00789
-
Rhythm Controllable and Efficient Zero-Shot Voice Conversion via Shortcut Flow Matching 1 Jun 2025 · 0 repositories · arXiv:2506.01014
-
A condensing approach to multiple shooting neural ordinary differential equation 31 May 2025 · 0 repositories · arXiv:2506.00724
-
Channel-Imposed Fusion: A Simple yet Effective Method for Medical Time Series Classification 31 May 2025 · 0 repositories · arXiv:2506.00337
-
Evaluating Robot Policies in a World Model 31 May 2025 · 0 repositories · arXiv:2506.00613
-
FinBERT2: A Specialized Bidirectional Encoder for Bridging the Gap in Finance-Specific Deployment of Large Language Models 31 May 2025 · 0 repositories · arXiv:2506.06335
-
Machine vs Machine: Using AI to Tackle Generative AI Threats in Assessment 31 May 2025 · 0 repositories · arXiv:2506.02046
-
Translate With Care: Addressing Gender Bias, Neutrality, and Reasoning in Large Language Model Translations 31 May 2025 · 1 repository · arXiv:2506.00748
-
Adversarial Threat Vectors and Risk Mitigation for Retrieval-Augmented Generation Systems 30 May 2025 · 0 repositories · arXiv:2506.00281
-
Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic Tasks 30 May 2025 · 1 repository · arXiv:2505.24876
-
ClueAnchor: Clue-Anchored Knowledge Reasoning Exploration and Optimization for Retrieval-Augmented Generation 30 May 2025 · 1 repository · arXiv:2505.24388
-
Cross-Attention Speculative Decoding 30 May 2025 · 0 repositories · arXiv:2505.24544
-
D2AF: A Dual-Driven Annotation and Filtering Framework for Visual Grounding 30 May 2025 · 0 repositories · arXiv:2505.24372
-
GradPower: Powering Gradients for Faster Language Model Pre-Training 30 May 2025 · 0 repositories · arXiv:2505.24275
-
Interpretable phenotyping of Heart Failure patients with Dutch discharge letters 30 May 2025 · 0 repositories · arXiv:2505.24619
-
Leveraging Intermediate Features of Vision Transformer for Face Anti-Spoofing 30 May 2025 · 0 repositories · arXiv:2505.24402
-
LightSAM: Parameter-Agnostic Sharpness-Aware Minimization 30 May 2025 · 0 repositories · arXiv:2505.24399
-
LPASS: Linear Probes as Stepping Stones for vulnerability detection using compressed LLMs 30 May 2025 · 0 repositories · arXiv:2505.24451
-
Mamba Knockout for Unraveling Factual Information Flow 30 May 2025 · 1 repository · arXiv:2505.24244Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples) · 1 pointer-only (licence)
-
Mastering Massive Multi-Task Reinforcement Learning via Mixture-of-Expert Decision Transformer 30 May 2025 · 1 repository · arXiv:2505.24378
-
MOFGPT: Generative Design of Metal-Organic Frameworks using Language Models 30 May 2025 · 1 repository · arXiv:2506.00198
-
PCIE_Pose Solution for EgoExo4D Pose and Proficiency Estimation Challenge 30 May 2025 · 0 repositories · arXiv:2505.24411
-
PersianMedQA: Language-Centric Evaluation of LLMs in the Persian Medical Domain 30 May 2025 · 0 repositories · arXiv:2506.00250
-
RealDrive: Retrieval-Augmented Driving with Diffusion Models 30 May 2025 · 0 repositories · arXiv:2505.24808
-
SPPSFormer: High-quality Superpoint-based Transformer for Roof Plane Instance Segmentation from Point Clouds 30 May 2025 · 0 repositories · arXiv:2505.24475
-
When GPT Spills the Tea: Comprehensive Assessment of Knowledge File Leakage in GPTs 30 May 2025 · 0 repositories · arXiv:2506.00197
-
Adversarial Semantic and Label Perturbation Attack for Pedestrian Attribute Recognition 29 May 2025 · 2 repositories · arXiv:2505.23313
-
ATLAS: Learning to Optimally Memorize the Context at Test Time 29 May 2025 · 0 repositories · arXiv:2505.23735
-
Bounded Rationality for LLMs: Satisficing Alignment at Inference-Time 29 May 2025 · 0 repositories · arXiv:2505.23729
-
Bridging the Gap Between Semantic and User Preference Spaces for Multi-modal Music Representation Learning 29 May 2025 · 0 repositories · arXiv:2505.23298
-
CF-DETR: Coarse-to-Fine Transformer for Real-Time Object Detection 29 May 2025 · 0 repositories · arXiv:2505.23317
-
Critical Batch Size Revisited: A Simple Empirical Approach to Large-Batch Language Model Training 29 May 2025 · 0 repositories · arXiv:2505.23971
-
DA-VPT: Semantic-Guided Visual Prompt Tuning for Vision Transformers 29 May 2025 · 1 repository · arXiv:2505.23694Syntology official (archive's flag): 4 ran · 4 ran (of which 4 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; every one of the 4 samples that ran constructed an object rather than computing a result (of 5 harvested samples)
-
Data-efficient Meta-models for Evaluation of Context-based Questions and Answers in LLMs 29 May 2025 · 0 repositories · arXiv:2505.23299
-
Daunce: Data Attribution through Uncertainty Estimation 29 May 2025 · 0 repositories · arXiv:2505.23223
-
Differential Gated Self-Attention 29 May 2025 · 0 repositories · arXiv:2505.24054
-
Enhancing LLM-Based Code Generation with Complexity Metrics: A Feedback-Driven Approach 29 May 2025 · 0 repositories · arXiv:2505.23953
-
Equivariant Spherical Transformer for Efficient Molecular Modeling 29 May 2025 · 0 repositories · arXiv:2505.23086
-
Evaluating AI capabilities in detecting conspiracy theories on YouTube 29 May 2025 · 1 repository · arXiv:2505.23570
-
From Images to Signals: Are Large Vision Models Useful for Time Series Analysis? 29 May 2025 · 0 repositories · arXiv:2505.24030
-
HyperPointFormer: Multimodal Fusion in 3D Space with Dual-Branch Cross-Attention Transformers 29 May 2025 · 1 repository · arXiv:2505.23206
-
Learning to Regulate: A New Event-Level Dataset of Capital Control Measures 29 May 2025 · 0 repositories · arXiv:2505.23025
-
Matryoshka Model Learning for Improved Elastic Student Models 29 May 2025 · 0 repositories · arXiv:2505.23337
-
MCP Safety Training: Learning to Refuse Falsely Benign MCP Exploits using Improved Preference Alignment 29 May 2025 · 0 repositories · arXiv:2505.23634
-
Patient Domain Supervised Contrastive Learning for Lung Sound Classification Using Mobile Phone 29 May 2025 · 0 repositories · arXiv:2505.23132
-
Probing Association Biases in LLM Moderation Over-Sensitivity 29 May 2025 · 0 repositories · arXiv:2505.23914
Tasks archive 2025-07-28
20 shown of 2,551 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
| Task | Papers |
|---|---|
| Language Modelling | 2,917 |
| Language Modeling | 2,286 |
| Retrieval | 1,734 |
| Question Answering | 1,420 |
| RAG | 1,290 |
| Sentence | 1,289 |
| Decoder | 1,284 |
| Retrieval-augmented Generation | 1,121 |
| Translation | 1,009 |
| Machine Translation | 905 |
| Large Language Model | 811 |
| Text Generation | 723 |
| Semantic Segmentation | 707 |
| Image Classification | 660 |
| Transfer Learning | 651 |
| Representation Learning | 596 |
| Sentiment Analysis | 595 |
| Object Detection | 585 |
| Text Classification | 559 |
| Classification | 554 |
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections