Methods › General › Stochastic Optimization › Adam
Adam
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Adam is an adaptive learning rate optimization algorithm that utilises both momentum and scaling, combining the benefits of RMSProp and SGD w/th Momentum. The optimizer is designed to be appropriate for non-stationary objectives and problems with very noisy and/or sparse gradients.
The weight updates are performed as:
wₜ = wₜ₋₁ - ηm̂ₜ/(√(v̂ₜ) + ϵ)
with
m̂ₜ = mₜ/(1-βᵗ₁)
v̂ₜ = vₜ/(1-βᵗ₂)
mₜ = β₁mₜ₋₁ + (1-β₁)gₜ
vₜ = β₂vₜ₋₁ + (1-β₂)gₜ²
η is the step size/learning rate, around 1e-3 in the original paper. ϵ is a small number, typically 1e-8 or 1e-10, to prevent dividing by zero. β₁ and β₂ are forgetting parameters, with typical values 0.9 and 0.999, respectively.
Papers archive 2025-07-28
30 shown of 24,390, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Constructing and Evaluating Declarative RAG Pipelines in PyTerrier 12 Jun 2025 · 1 repository · arXiv:2506.10802
-
PyLO: Towards Accessible Learned Optimizers in PyTorch 12 Jun 2025 · 1 repository · arXiv:2506.10315
-
Towards Robust Multimodal Emotion Recognition under Missing Modalities and Distribution Shifts 12 Jun 2025 · 1 repository · arXiv:2506.10452
-
A Novel Lightweight Transformer with Edge-Aware Fusion for Remote Sensing Image Captioning 11 Jun 2025 · 0 repositories · arXiv:2506.09429
-
Auto-Compressing Networks 11 Jun 2025 · 0 repositories · arXiv:2506.09714
-
Learning Efficient and Generalizable Graph Retriever for Knowledge-Graph Question Answering 11 Jun 2025 · 1 repository · arXiv:2506.09645
-
SparseSSM: Efficient Selective Structured State Space Models Can Be Pruned in One-Shot 11 Jun 2025 · 0 repositories · arXiv:2506.09613
-
Online Learning-guided Learning Rate Adaptation via Gradient Alignment 10 Jun 2025 · 0 repositories · arXiv:2506.08419
-
ADAM: Autonomous Discovery and Annotation Model using LLMs for Context-Aware Annotations 10 Jun 2025 · 0 repositories · arXiv:2506.08968
-
An Adaptive Method Stabilizing Activations for Enhanced Generalization 10 Jun 2025 · 1 repository · arXiv:2506.08353
-
FZOO: Fast Zeroth-Order Optimizer for Fine-Tuning Large Language Models towards Adam-Scale Speed 10 Jun 2025 · 0 repositories · arXiv:2506.09034
-
Hierarchical Neural Collapse Detection Transformer for Class Incremental Object Detection 10 Jun 2025 · 0 repositories · arXiv:2506.08562
-
Hyperspectral Image Classification via Transformer-based Spectral-Spatial Attention Decoupling and Adaptive Gating 10 Jun 2025 · 1 repository · arXiv:2506.08324
-
MedMoE: Modality-Specialized Mixture of Experts for Medical Vision-Language Understanding 10 Jun 2025 · 0 repositories · arXiv:2506.08356
-
MetaTT: A Global Tensor-Train Adapter for Parameter-Efficient Fine-Tuning 10 Jun 2025 · 0 repositories · arXiv:2506.09105
-
PatchGuard: Adversarially Robust Anomaly Detection and Localization through Vision Transformers and Pseudo Anomalies 10 Jun 2025 · 2 repositories · arXiv:2506.09237
-
Robust Visual Localization via Semantic-Guided Multi-Scale Transformer 10 Jun 2025 · 0 repositories · arXiv:2506.08526
-
TACTIC: Translation Agents with Cognitive-Theoretic Interactive Collaboration 10 Jun 2025 · 1 repository · arXiv:2506.08403
-
4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos 9 Jun 2025 · 0 repositories · arXiv:2506.08015
-
Lightweight Sequential Transformers for Blood Glucose Level Prediction in Type-1 Diabetes 9 Jun 2025 · 0 repositories · arXiv:2506.07864
-
LlamaRec-LKG-RAG: A Single-Pass, Learnable Knowledge Graph-RAG Framework for LLM-Based Ranking 9 Jun 2025 · 1 repository · arXiv:2506.07449
-
LLM-driven Indoor Scene Layout Generation via Scaled Human-aligned Data Synthesis and Multi-Stage Preference Optimization 9 Jun 2025 · 0 repositories · arXiv:2506.07570
-
MADFormer: Mixed Autoregressive and Diffusion Transformers for Continuous Image Generation 9 Jun 2025 · 0 repositories · arXiv:2506.07999
-
Quantum Graph Transformer for NLP Sentiment Classification 9 Jun 2025 · 0 repositories · arXiv:2506.07937
-
SceneRAG: Scene-level Retrieval-Augmented Generation for Video Understanding 9 Jun 2025 · 0 repositories · arXiv:2506.07600
-
Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models 8 Jun 2025 · 0 repositories · arXiv:2506.07121
-
Adam assisted Fully informed Particle Swarm Optimzation ( Adam-FIPSO ) based Parameter Prediction for the Quantum Approximate Optimization Algorithm (QAOA) 7 Jun 2025 · 0 repositories · arXiv:2506.06790
-
Breaking Data Silos: Towards Open and Scalable Mobility Foundation Models via Generative Continual Learning 7 Jun 2025 · 0 repositories · arXiv:2506.06694
-
Can In-Context Reinforcement Learning Recover From Reward Poisoning Attacks? 7 Jun 2025 · 0 repositories · arXiv:2506.06891
-
BEAST: Efficient Tokenization of B-Splines Encoded Action Sequences for Imitation Learning 6 Jun 2025 · 0 repositories · arXiv:2506.06072
Tasks archive 2025-07-28
20 shown of 2,551 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
| Task | Papers |
|---|---|
| Language Modelling | 2,917 |
| Language Modeling | 2,286 |
| Retrieval | 1,734 |
| Question Answering | 1,420 |
| RAG | 1,290 |
| Sentence | 1,289 |
| Decoder | 1,284 |
| Retrieval-augmented Generation | 1,121 |
| Translation | 1,009 |
| Machine Translation | 905 |
| Large Language Model | 811 |
| Text Generation | 723 |
| Semantic Segmentation | 707 |
| Image Classification | 660 |
| Transfer Learning | 651 |
| Representation Learning | 596 |
| Sentiment Analysis | 595 |
| Object Detection | 585 |
| Text Classification | 559 |
| Classification | 554 |
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections