Papers › Robust and On-the-fly Dataset Denoising for Image Classification
Robust and On-the-fly Dataset Denoising for Image Classification
Jiaming Song, Lunjia Hu, Michael Auli, Yann Dauphin, Tengyu Ma
Memorization in over-parameterized neural networks could severely hurt generalization in the presence of mislabeled examples. However, mislabeled examples are hard to avoid in extremely large datasets collected with weak supervision. We address this problem by reasoning counterfactually about the loss distribution of examples with uniform random labels had they were trained with the real examples, and use this information to remove noisy examples from the training set. First, we observe that examples with uniform random labels have higher losses when trained with stochastic gradient descent under large learning rates. Then, we propose to model the loss distribution of the counterfactual examples using only the network parameters, which is able to model such examples with remarkable success. Finally, we propose to remove examples whose loss exceeds a certain quantile of the modeled loss distribution. This leads to On-the-fly Data Denoising (ODD), a simple yet effective algorithm that is robust to mislabeled examples, while introducing almost zero computational overhead compared to standard training. ODD is able to achieve state-of-the-art results on a wide range of datasets including real-world ones such as WebVision and Clothing1M.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
1 archive task tag without a task page not shown.
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Image Classification | mini WebVision 1.0 | ODD (Inception-ResNet-v2) | ImageNet Top-1 Accuracy | 66.7 | #37 of 47 | Archive leaderboard | report |
| Image Classification | mini WebVision 1.0 | ODD (Inception-ResNet-v2) | ImageNet Top-5 Accuracy | 86.3 | #37 of 47 | Archive leaderboard | report |
| Image Classification | mini WebVision 1.0 | ODD (Inception-ResNet-v2) | Top-1 Accuracy | 74.6 | #37 of 47 | Archive leaderboard | report |
| Image Classification | mini WebVision 1.0 | ODD (Inception-ResNet-v2) | Top-5 Accuracy | 90.6 | #37 of 47 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections