Methods › General › Stochastic Optimization › RMSProp
RMSProp
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
RMSProp is an unpublished adaptive learning rate optimizer proposed by Geoff Hinton. The motivation is that the magnitude of gradients can differ for different weights, and can change during learning, making it hard to choose a single global learning rate. RMSProp tackles this by keeping a moving average of the squared gradient and adjusting the weight updates by this magnitude. The gradient updates are performed as:
E[g²]ₜ = γE[g²]ₜ₋₁ + (1 - γ) g²ₜ
θₜ₊₁ = θₜ - η/(√(E[g²]ₜ + ϵ))gₜ
Hinton suggests γ=0.9, with a good default for η as 0.001.
Image: Alec Radford
In Syntology name not resolved (unknown_name). The lookup matched no method, paper or venue by this name; that asserts nothing about whether code exists.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 519 · with a code link: 236 · where Syntology ran a sample: 70 (61 with a run with no instrument failure, 9 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (70 of 519 tagged: 61 with a run with no instrument failure, 9 where every run was a failure of Syntology's instrument)
Page 1 of 6: papers 1 to 100 of 519, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
Hindsight-Guided Momentum (HGM) Optimizer: An Approach to Adaptive Learning Rate 22 Jun 2025 · 0 repositories · arXiv:2506.22479
-
Deep Learning-Based BMD Estimation from Radiographs with Conformal Uncertainty Quantification 28 May 2025 · 0 repositories · arXiv:2505.22551
-
Intelligent Incident Hypertension Prediction in Obstructive Sleep Apnea 27 May 2025 · 0 repositories · arXiv:2505.20615
-
Deep Learning for Breast Cancer Detection: Comparative Analysis of ConvNeXT and EfficientNet 24 May 2025 · 0 repositories · arXiv:2505.18725
-
SuperPure: Efficient Purification of Localized and Distributed Adversarial Patches via Super-Resolution GAN Models 22 May 2025 · 0 repositories · arXiv:2505.16318
-
Vulnerability of Transfer-Learned Neural Networks to Data Reconstruction Attacks in Small-Data Regime 20 May 2025 · 1 repository · arXiv:2505.14323
-
Defect Detection in Photolithographic Patterns Using Deep Learning Models Trained on Synthetic Data 15 May 2025 · 0 repositories · arXiv:2505.10192
-
Real-World fNIRS-Based Brain-Computer Interfaces: Benchmarking Deep Learning and Classical Models in Interactive Gaming 15 May 2025 · 0 repositories · arXiv:2505.10536
-
Trial and Trust: Addressing Byzantine Attacks with Comprehensive Defense Strategy 12 May 2025 · 0 repositories · arXiv:2505.07614
-
V-EfficientNets: Vector-Valued Efficiently Scaled Convolutional Neural Network Models 8 May 2025 · 1 repository · arXiv:2505.05659
-
AnomalyMatch: Discovering Rare Objects of Interest with Semi-supervised and Active Learning 6 May 2025 · 1 repository · arXiv:2505.03509
-
CSASN: A Multitask Attention-Based Framework for Heterogeneous Thyroid Carcinoma Classification in Ultrasound Images 4 May 2025 · 0 repositories · arXiv:2505.02211
-
Conformal Prediction for Indoor Positioning with Correctness Coverage Guarantees 3 May 2025 · 0 repositories · arXiv:2505.01810
-
Sharp higher order convergence rates for the Adam optimizer 28 Apr 2025 · 0 repositories · arXiv:2504.19426
-
Some Optimizers are More Equal: Understanding the Role of Optimizers in Group Fairness 21 Apr 2025 · 0 repositories · arXiv:2504.14882
-
Covariant Gradient Descent 7 Apr 2025 · 0 repositories · arXiv:2504.05279
-
Training Frozen Feature Pyramid DINOv2 for Eyelid Measurements with Infinite Encoding and Orthogonal Regularization 1 Apr 2025 · 1 repository · arXiv:2504.00515
-
Efficient Building Roof Type Classification: A Domain-Specific Self-Supervised Approach 28 Mar 2025 · 0 repositories · arXiv:2503.22251
-
World Model Agents with Change-Based Intrinsic Motivation 26 Mar 2025 · 1 repository · arXiv:2503.21047
-
Deep learning-based identification of precipitation clouds from all-sky camera data for observatory safety 24 Mar 2025 · 0 repositories · arXiv:2503.18670
-
Weakly Supervised Pixel-Level Annotation with Visual Interpretability 25 Feb 2025 · 0 repositories · arXiv:2502.17824
-
Reducing false positives in strong lens detection through effective augmentation and ensemble learning 20 Feb 2025 · 0 repositories · arXiv:2502.14936
-
CAE-Net: Generalized Deepfake Image Detection using Convolution and Attention Mechanisms with Spatial and Frequency Domain Features 15 Feb 2025 · 0 repositories · arXiv:2502.10682
-
AI-Driven Solutions for Falcon Disease Classification: Concatenated ConvNeXt cum EfficientNet AI Model Approach 7 Feb 2025 · 0 repositories · arXiv:2502.04682
-
Enhancing Brain Tumor Segmentation Using Channel Attention and Transfer learning 19 Jan 2025 · 1 repository · arXiv:2501.11196
-
IoT-Based Real-Time Medical-Related Human Activity Recognition Using Skeletons and Multi-Stage Deep Learning for Healthcare 13 Jan 2025 · 1 repository · arXiv:2501.07039
-
LayerDropBack: A Universally Applicable Approach for Accelerating Training of Deep Networks 23 Dec 2024 · 1 repository · arXiv:2412.18027
-
Comparative Analysis of Deep Learning Approaches for Harmful Brain Activity Detection Using EEG 10 Dec 2024 · 0 repositories · arXiv:2412.07878
-
Impact of Privacy Parameters on Deep Learning Models for Image Classification 9 Dec 2024 · 0 repositories · arXiv:2412.06689
-
Thermal Image-based Fault Diagnosis in Induction Machines via Self-Organized Operational Neural Networks 8 Dec 2024 · 0 repositories · arXiv:2412.05901
-
Adaptive Optimization for Enhanced Efficiency in Large-Scale Language Model Training 6 Dec 2024 · 0 repositories · arXiv:2412.04718
-
Pairwise Discernment of AffectNet Expressions with ArcFace 1 Dec 2024 · 0 repositories · arXiv:2412.01860
-
Breast Tumor Classification Using EfficientNet Deep Learning Model 26 Nov 2024 · 1 repository · arXiv:2411.17870
-
Explainable AI Approach using Near Misses Analysis 25 Nov 2024 · 0 repositories · arXiv:2411.16895
-
TransFair: Transferring Fairness from Ocular Disease Classification to Progression Prediction 24 Nov 2024 · 0 repositories · arXiv:2412.00051
-
Classification of Geographical Land Structure Using Convolution Neural Network and Transfer Learning 19 Nov 2024 · 0 repositories · arXiv:2411.12415
-
Deep Learning-Driven Heat Map Analysis for Evaluating thickness of Wounded Skin Layers 19 Nov 2024 · 0 repositories · arXiv:2411.12678
-
Deep Learning-Based Prediction of PET Amyloid Status Using Multi-Contrast MRI 18 Nov 2024 · 0 repositories · arXiv:2411.12061
-
Modeling AdaGrad, RMSProp, and Adam with Integro-Differential Equations 14 Nov 2024 · 1 repository · arXiv:2411.09734Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
SAG-ViT: A Scale-Aware, High-Fidelity Patching Approach with Graph Attention for Vision Transformers 14 Nov 2024 · 1 repository · arXiv:2411.09420
-
A Composite Fault Diagnosis Model for NPPs Based on Bayesian-EfficientNet Module 13 Nov 2024 · 0 repositories · arXiv:2411.17707
-
Research on fault diagnosis of nuclear power first-second circuit based on hierarchical multi-granularity classification network 12 Nov 2024 · 0 repositories · arXiv:2411.07453
-
Advancing Meteorological Forecasting: AI-based Approach to Synoptic Weather Map Analysis 8 Nov 2024 · 0 repositories · arXiv:2411.05384
-
Understanding Optimization in Deep Learning with Central Flows 31 Oct 2024 · 0 repositories · arXiv:2410.24206Syntology 7 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 7 harvested samples) · 7 pointer-only (licence)
-
Breast Cancer Histopathology Classification using CBAM-EfficientNetV2 with Transfer Learning 29 Oct 2024 · 0 repositories · arXiv:2410.22392
-
Exploring Self-Supervised Learning with U-Net Masked Autoencoders and EfficientNet B7 for Improved Classification 25 Oct 2024 · 1 repository · arXiv:2410.19899
-
Efficient Feature Extraction Using Light-Weight CNN Attention-Based Deep Learning Architectures for Ultrasound Fetal Plane Classification 22 Oct 2024 · 0 repositories · arXiv:2410.17396
-
Osteoporosis screening: Leveraging EfficientNet with complete and cropped facial panoramic radiography imaging 18 Oct 2024 · 1 repository
-
Non-convergence to global minimizers in data driven supervised deep learning: Adam and stochastic gradient descent optimization provably fail to converge to global minimizers in the training of deep neural networks with ReLU activation 14 Oct 2024 · 0 repositories · arXiv:2410.10533
-
The Epochal Sawtooth Effect: Unveiling Training Loss Oscillations in Adam and Other Optimizers 14 Oct 2024 · 1 repository · arXiv:2410.10056Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples) · 3 pointer-only (licence)
-
Comparison of Machine Learning Approaches for Classifying Spinodal Events 13 Oct 2024 · 0 repositories · arXiv:2410.09756
-
Deep Learning Ensemble for Predicting Diabetic Macular Edema Onset Using Ultra-Wide Field Color Fundus Image 9 Oct 2024 · 0 repositories · arXiv:2410.06483
-
A second-order-like optimizer with adaptive gradient scaling for deep learning 8 Oct 2024 · 1 repository · arXiv:2410.05871
-
Art Forgery Detection using Kolmogorov Arnold and Convolutional Neural Networks 7 Oct 2024 · 0 repositories · arXiv:2410.04866
-
The Comparison of Individual Cat Recognition Using Neural Networks 3 Oct 2024 · 0 repositories · arXiv:2410.02305
-
Preconditioning for Accelerated Gradient Descent Optimization and Regularization 30 Sep 2024 · 0 repositories · arXiv:2410.00232
-
Automated Disease Diagnosis in Pumpkin Plants Using Advanced CNN Models 29 Sep 2024 · 0 repositories · arXiv:2410.00062
-
Multi-Model Ensemble Approach for Accurate Bi-Atrial Segmentation in LGE-MRI of Atrial Fibrillation Patients 24 Sep 2024 · 0 repositories · arXiv:2409.16083
-
Deep Learning and Machine Learning, Advancing Big Data Analytics and Management: Tensorflow Pretrained Models 20 Sep 2024 · 0 repositories · arXiv:2409.13566
-
Deep Learning-Based Detection of Referable Diabetic Retinopathy and Macular Edema Using Ultra-Widefield Fundus Imaging 19 Sep 2024 · 0 repositories · arXiv:2409.12854
-
Recognition of Harmful Phytoplankton from Microscopic Images using Deep Learning 19 Sep 2024 · 0 repositories · arXiv:2409.12900
-
Less Memory Means smaller GPUs: Backpropagation with Compressed Activations 18 Sep 2024 · 0 repositories · arXiv:2409.11902
-
Application of Langevin Dynamics to Advance the Quantum Natural Gradient Optimization Algorithm 3 Sep 2024 · 1 repository · arXiv:2409.01978
-
SFA-Net: Semantic Feature Adjustment Network for Remote Sensing Image Segmentation 3 Sep 2024 · 1 repository
-
Evaluating Deep Learning Models for Breast Cancer Classification: A Comparative Study 29 Aug 2024 · 2 repositories · arXiv:2408.16859
-
First line of defense: A robust first layer mitigates adversarial attacks 21 Aug 2024 · 1 repository · arXiv:2408.11680
-
Towards Robust Federated Image Classification: An Empirical Study of Weight Selection Strategies in Manufacturing 19 Aug 2024 · 0 repositories · arXiv:2408.10024
-
Efficient Deep Model-Based Optoacoustic Image Reconstruction 13 Aug 2024 · 0 repositories · arXiv:2408.07109
-
Advancing Vietnamese Visual Question Answering with Transformer and Convolutional Integration 30 Jul 2024 · 1 repository · arXiv:2407.21229
-
Enhanced Deep Learning Methodologies and MRI Selection Techniques for Dementia Diagnosis in the Elderly Population 24 Jul 2024 · 0 repositories · arXiv:2407.17324
-
Wavelet-based Autoencoder and EfficientNet for Schizophrenia Detection from EEG Signals 24 Jul 2024 · 0 repositories · arXiv:2407.17540
-
EfficientCD: A New Strategy For Change Detection Based With Bi-temporal Layers Exchanged 22 Jul 2024 · 1 repository · arXiv:2407.15999
-
Hybrid Deep Learning Framework for Enhanced Melanoma Detection 16 Jul 2024 · 0 repositories · arXiv:2408.00772
-
Improving Dental Diagnostics: Enhanced Convolution with Spatial Attention Mechanism 11 Jul 2024 · 0 repositories · arXiv:2407.08114
-
Non-convergence of Adam and other adaptive stochastic gradient descent optimization methods for non-vanishing learning rates 11 Jul 2024 · 0 repositories · arXiv:2407.08100
-
Predicting Heart Failure with Attention Learning Techniques Utilizing Cardiovascular Data 11 Jul 2024 · 0 repositories · arXiv:2407.08289
-
Explainable Differential Privacy-Hyperdimensional Computing for Balancing Privacy and Transparency in Additive Manufacturing Monitoring 9 Jul 2024 · 1 repository · arXiv:2407.07066
-
Analysis of Modern Computer Vision Models for Blood Cell Classification 30 Jun 2024 · 1 repository · arXiv:2407.00759
-
How to Learn More? Exploring Kolmogorov-Arnold Networks for Hyperspectral Image Classification 22 Jun 2024 · 1 repository · arXiv:2406.15719
-
Liveness Detection in Computer Vision: Transformer-based Self-Supervised Learning for Face Anti-Spoofing 19 Jun 2024 · 0 repositories · arXiv:2406.13860
-
Online-Adaptive Anomaly Detection for Defect Identification in Aircraft Assembly 18 Jun 2024 · 0 repositories · arXiv:2406.12698
-
Which Backbone to Use: A Resource-efficient Domain Specific Comparison for Computer Vision 9 Jun 2024 · 1 repository · arXiv:2406.05612
-
U-Net Ensemble for Enhanced Semantic Segmentation in Remote Sensing Imagery 8 Jun 2024 · 0 repositories
-
A Diagnostic Model for Acute Lymphoblastic Leukemia Using Metaheuristics and Deep Learning Methods 2 Jun 2024 · 0 repositories · arXiv:2406.18568
-
Kolmogorov-Arnold Network for Satellite Image Classification in Remote Sensing 2 Jun 2024 · 1 repository · arXiv:2406.00600
-
Local Methods with Adaptivity via Scaling 2 Jun 2024 · 0 repositories · arXiv:2406.00846
-
Credal Wrapper of Model Averaging for Uncertainty Estimation in Classification 23 May 2024 · 0 repositories · arXiv:2405.15047
-
Surge Phenomenon in Optimal Learning Rate and Batch Size Scaling 23 May 2024 · 0 repositories · arXiv:2405.14578
-
Enhancing the analysis of murine neonatal ultrasonic vocalizations: Development, evaluation, and application of different mathematical models 17 May 2024 · 1 repository · arXiv:2405.12957
-
Revolutionizing Traffic Sign Recognition: Unveiling the Potential of Vision Transformers 29 Apr 2024 · 0 repositories · arXiv:2404.19066
-
Guided AbsoluteGrad: Magnitude of Gradients Matters to Explanation's Localization and Saliency 23 Apr 2024 · 1 repository · arXiv:2404.15564
-
Depth Estimation using Weighted-loss and Transfer Learning 11 Apr 2024 · 0 repositories · arXiv:2404.07686
-
Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance 1 Apr 2024 · 0 repositories · arXiv:2404.01436
-
COVID-19 detection from pulmonary CT scans using a novel EfficientNet with attention mechanism 18 Mar 2024 · 1 repository · arXiv:2403.11505
-
HSEmotion Team at the 6th ABAW Competition: Facial Expressions, Valence-Arousal and Emotion Intensity Prediction 18 Mar 2024 · 0 repositories · arXiv:2403.11590
-
Quantization Effects on Neural Networks Perception: How would quantization change the perceptual field of vision models? 15 Mar 2024 · 1 repository · arXiv:2403.09939
-
Multiscale Low-Frequency Memory Network for Improved Feature Extraction in Convolutional Neural Networks 13 Mar 2024 · 1 repository · arXiv:2403.08157
-
Non-asymptotic Analysis of Biased Adaptive Stochastic Approximation 5 Feb 2024 · 1 repository · arXiv:2402.02857
-
Survival and grade of the glioma prediction using transfer learning 4 Feb 2024 · 0 repositories · arXiv:2402.03384
-
On the O((√(d))/(T^(1/4))) Convergence Rate of RMSProp and Its Momentum Extension Measured by ℓ₁ Norm 1 Feb 2024 · 0 repositories · arXiv:2402.00389
Tasks archive 2025-07-28
20 shown of 353 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections