Papers › Revisiting ResNets: Improved Training and Scaling Strategies
Revisiting ResNets: Improved Training and Scaling Strategies
Irwan Bello, William Fedus, Xianzhi Du, Ekin D. Cubuk, Aravind Srinivas, Tsung-Yi Lin, Jonathon Shlens, Barret Zoph
Novel computer vision architectures monopolize the spotlight, but the impact of the model architecture is often conflated with simultaneous changes to training methodology and scaling strategies. Our work revisits the canonical ResNet (He et al., 2015) and studies these three aspects in an effort to disentangle them. Perhaps surprisingly, we find that training and scaling strategies may matter more than architectural changes, and further, that the resulting ResNets match recent state-of-the-art models. We show that the best performing scaling strategy depends on the training regime and offer two new scaling strategies: (1) scale model depth in regimes where overfitting can occur (width scaling is preferable otherwise); (2) increase image resolution more slowly than previously recommended (Tan & Le, 2019). Using improved training and scaling strategies, we design a family of ResNet architectures, ResNet-RS, which are 1.7x - 2.7x faster than EfficientNets on TPUs, while achieving similar accuracies on ImageNet. In a large-scale semi-supervised learning setup, ResNet-RS achieves 86.2% top-1 ImageNet accuracy, while being 4.7x faster than EfficientNet NoisyStudent. The training techniques improve transfer performance on a suite of downstream tasks (rivaling state-of-the-art self-supervised algorithms) and extend to video classification on Kinetics-400. We recommend practitioners use these simple revised ResNets as baselines for future research.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Document Image Classification | AIP | ResNet-RS (ResNet-200 + RS training tricks) | Top 1 Accuracy - Verb | 83.4 | #1 of 1 | Archive leaderboard | report |
| Image Classification | ImageNet | ResNet-RS-50 (160 image res) | GFLOPs | 4.6 | #322 of 1060 | Archive leaderboard | report |
| Image Classification | ImageNet | ResNet-RS-50 (160 image res) | Number of params | 192M | #322 of 1060 | Archive leaderboard | report |
| Image Classification | ImageNet | ResNet-RS-50 (160 image res) | Top 1 Accuracy | 84.4% | #322 of 1060 | Archive leaderboard | report |
| Image Classification | ImageNet | ResNet-RS-270 (256 image res) | GFLOPs | 54 | #384 of 1060 | Archive leaderboard | report |
| Image Classification | ImageNet | ResNet-RS-270 (256 image res) | Top 1 Accuracy | 83.8% | #384 of 1060 | Archive leaderboard | report |
| Image Classification | PRImA | ResNet-152 2x (RS training) | Percentage correct | 89.3 | #1 of 1 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Introduced by this paper: ResNet-RS
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections