Papers › Modified Step Size for Enhanced Stochastic Gradient Descent: Convergence and Experiments
Modified Step Size for Enhanced Stochastic Gradient Descent: Convergence and Experiments
M. Soheil Shamaee, S. Fathi Hafshejani
This paper introduces a novel approach to enhance the performance of the stochastic gradient descent (SGD) algorithm by incorporating a modified decay step size based on 1/(√(t)). The proposed step size integrates a logarithmic term, leading to the selection of smaller values in the final iterations. Our analysis establishes a convergence rate of O(lnT/(√(T))) for smooth non-convex functions without the Polyak-{\L}ojasiewicz condition. To evaluate the effectiveness of our approach, we conducted numerical experiments on image classification tasks using the FashionMNIST, and CIFAR10 datasets, and the results demonstrate significant improvements in accuracy, with enhancements of 0.5% and 1.4% observed, respectively, compared to the traditional 1/(√(t)) step size. The source code can be found at \\\url{https://github.com/Shamaeem/LNSQRTStepSize}.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
No leaderboard rows for this paper in the archive.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections