Methods › General › Stochastic Optimization › RMSProp
RMSProp
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
RMSProp is an unpublished adaptive learning rate optimizer proposed by Geoff Hinton. The motivation is that the magnitude of gradients can differ for different weights, and can change during learning, making it hard to choose a single global learning rate. RMSProp tackles this by keeping a moving average of the squared gradient and adjusting the weight updates by this magnitude. The gradient updates are performed as:
E[g²]ₜ = γE[g²]ₜ₋₁ + (1 - γ) g²ₜ
θₜ₊₁ = θₜ - η/(√(E[g²]ₜ + ϵ))gₜ
Hinton suggests γ=0.9, with a good default for η as $0.001$.
Image: Alec Radford
Papers archive 2025-07-28
30 shown of 519, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Hindsight-Guided Momentum (HGM) Optimizer: An Approach to Adaptive Learning Rate 22 Jun 2025 · 0 repositories · arXiv:2506.22479
-
Deep Learning-Based BMD Estimation from Radiographs with Conformal Uncertainty Quantification 28 May 2025 · 0 repositories · arXiv:2505.22551
-
Intelligent Incident Hypertension Prediction in Obstructive Sleep Apnea 27 May 2025 · 0 repositories · arXiv:2505.20615
-
Deep Learning for Breast Cancer Detection: Comparative Analysis of ConvNeXT and EfficientNet 24 May 2025 · 0 repositories · arXiv:2505.18725
-
SuperPure: Efficient Purification of Localized and Distributed Adversarial Patches via Super-Resolution GAN Models 22 May 2025 · 0 repositories · arXiv:2505.16318
-
Vulnerability of Transfer-Learned Neural Networks to Data Reconstruction Attacks in Small-Data Regime 20 May 2025 · 1 repository · arXiv:2505.14323
-
Defect Detection in Photolithographic Patterns Using Deep Learning Models Trained on Synthetic Data 15 May 2025 · 0 repositories · arXiv:2505.10192
-
Real-World fNIRS-Based Brain-Computer Interfaces: Benchmarking Deep Learning and Classical Models in Interactive Gaming 15 May 2025 · 0 repositories · arXiv:2505.10536
-
Trial and Trust: Addressing Byzantine Attacks with Comprehensive Defense Strategy 12 May 2025 · 0 repositories · arXiv:2505.07614
-
V-EfficientNets: Vector-Valued Efficiently Scaled Convolutional Neural Network Models 8 May 2025 · 1 repository · arXiv:2505.05659
-
AnomalyMatch: Discovering Rare Objects of Interest with Semi-supervised and Active Learning 6 May 2025 · 1 repository · arXiv:2505.03509
-
CSASN: A Multitask Attention-Based Framework for Heterogeneous Thyroid Carcinoma Classification in Ultrasound Images 4 May 2025 · 0 repositories · arXiv:2505.02211
-
Conformal Prediction for Indoor Positioning with Correctness Coverage Guarantees 3 May 2025 · 0 repositories · arXiv:2505.01810
-
Sharp higher order convergence rates for the Adam optimizer 28 Apr 2025 · 0 repositories · arXiv:2504.19426
-
Some Optimizers are More Equal: Understanding the Role of Optimizers in Group Fairness 21 Apr 2025 · 0 repositories · arXiv:2504.14882
-
Covariant Gradient Descent 7 Apr 2025 · 0 repositories · arXiv:2504.05279
-
Training Frozen Feature Pyramid DINOv2 for Eyelid Measurements with Infinite Encoding and Orthogonal Regularization 1 Apr 2025 · 1 repository · arXiv:2504.00515
-
Efficient Building Roof Type Classification: A Domain-Specific Self-Supervised Approach 28 Mar 2025 · 0 repositories · arXiv:2503.22251
-
World Model Agents with Change-Based Intrinsic Motivation 26 Mar 2025 · 1 repository · arXiv:2503.21047
-
Deep learning-based identification of precipitation clouds from all-sky camera data for observatory safety 24 Mar 2025 · 0 repositories · arXiv:2503.18670
-
Weakly Supervised Pixel-Level Annotation with Visual Interpretability 25 Feb 2025 · 0 repositories · arXiv:2502.17824
-
Reducing false positives in strong lens detection through effective augmentation and ensemble learning 20 Feb 2025 · 0 repositories · arXiv:2502.14936
-
CAE-Net: Generalized Deepfake Image Detection using Convolution and Attention Mechanisms with Spatial and Frequency Domain Features 15 Feb 2025 · 0 repositories · arXiv:2502.10682
-
AI-Driven Solutions for Falcon Disease Classification: Concatenated ConvNeXt cum EfficientNet AI Model Approach 7 Feb 2025 · 0 repositories · arXiv:2502.04682
-
Enhancing Brain Tumor Segmentation Using Channel Attention and Transfer learning 19 Jan 2025 · 1 repository · arXiv:2501.11196
-
IoT-Based Real-Time Medical-Related Human Activity Recognition Using Skeletons and Multi-Stage Deep Learning for Healthcare 13 Jan 2025 · 1 repository · arXiv:2501.07039
-
LayerDropBack: A Universally Applicable Approach for Accelerating Training of Deep Networks 23 Dec 2024 · 1 repository · arXiv:2412.18027
-
Comparative Analysis of Deep Learning Approaches for Harmful Brain Activity Detection Using EEG 10 Dec 2024 · 0 repositories · arXiv:2412.07878
-
Impact of Privacy Parameters on Deep Learning Models for Image Classification 9 Dec 2024 · 0 repositories · arXiv:2412.06689
-
Thermal Image-based Fault Diagnosis in Induction Machines via Self-Organized Operational Neural Networks 8 Dec 2024 · 0 repositories · arXiv:2412.05901
Tasks archive 2025-07-28
20 shown of 353 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections