</> Code with Papers by Syntology

Papers › Escaping Saddle Points for Effective Generalization on Class-Imbalanced Data

Escaping Saddle Points for Effective Generalization on Class-Imbalanced Data

28 Dec 2022arXiv:2212.13827archive 2025-07-28

Harsh Rangwani, Sumukh K Aithal, Mayank Mishra, R. Venkatesh Babu

Real-world datasets exhibit imbalances of varying types and degrees. Several techniques based on re-weighting and margin adjustment of loss are often used to enhance the performance of neural networks, particularly on minority classes. In this work, we analyze the class-imbalanced learning problem by examining the loss landscape of neural networks trained with re-weighting and margin-based techniques. Specifically, we examine the spectral density of Hessian of class-wise loss, through which we observe that the network weights converge to a saddle point in the loss landscapes of minority classes. Following this observation, we also find that optimization methods designed to escape from saddle points can be effectively used to improve generalization on minority classes. We further theoretically and empirically demonstrate that Sharpness-Aware Minimization (SAM), a recent technique that encourages convergence to a flat minima, can be effectively used to escape saddle points for minority classes. Using SAM results in a 6.2\% increase in accuracy on the minority classes over the state-of-the-art Vector Scaling Loss, leading to an overall average increase of 4\% across imbalanced datasets. The code is available at: https://github.com/val-iisc/Saddle-LongTail.

PaperPDFCode

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

val-iisc/saddle-longtail officialmentioned in papermentioned on GitHubpytorchMIT report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Long-tail Learning

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Long-tail Learning CIFAR-10-LT (ρ=10) LDAM + DRW + SAM Error Rate 10.6 #28 of 50 Archive leaderboard report
Long-tail Learning CIFAR-10-LT (ρ=100) GLMC + SAM Error Rate 10.82 #2 of 28 Archive leaderboard report
Long-tail Learning CIFAR-10-LT (ρ=100) VS + SAM Error Rate 17.6 #17 of 28 Archive leaderboard report
Long-tail Learning CIFAR-10-LT (ρ=200) LDAM + DRW + SAM Error Rate 21.9 #1 of 2 Archive leaderboard report
Long-tail Learning CIFAR-10-LT (ρ=50) GLMC + SAM Error Rate 8.44 #1 of 8 Archive leaderboard report
Long-tail Learning CIFAR-100-LT (ρ=100) GLMC + SAM Error Rate 40.99 #6 of 66 Archive leaderboard report
Long-tail Learning CIFAR-100-LT (ρ=100) PaCo + SAM Error Rate 47.0 #21 of 66 Archive leaderboard report
Long-tail Learning CIFAR-100-LT (ρ=100) VS + SAM Error Rate 53.4 #41 of 66 Archive leaderboard report
Long-tail Learning CIFAR-100-LT (ρ=200) PaCo + SAM Error Rate 52.0 #1 of 2 Archive leaderboard report
Long-tail Learning CIFAR-100-LT (ρ=50) GLMC + SAM Error Rate 34.72 #5 of 25 Archive leaderboard report
Long-tail Learning ImageNet-LT LDAM + DRW + SAM Top-1 Accuracy 53.1 #45 of 69 Archive leaderboard report
Long-tail Learning iNaturalist 2018 LDAM + DRW + SAM Top-1 Accuracy 70.1 #34 of 43 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Sharpness-Aware Minimization

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections