{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/densenets-reloaded-paradigm-shift-beyond","title":"DenseNets Reloaded: Paradigm Shift Beyond ResNets and ViTs","arxiv_id":"2403.19588","date":"2024-03-28","proceeding":null,"authors":["Donghyun Kim","Byeongho Heo","Dongyoon Han"],"abstract":"This paper revives Densely Connected Convolutional Networks (DenseNets) and reveals the underrated effectiveness over predominant ResNet-style architectures. We believe DenseNets' potential was overlooked due to untouched training methods and traditional design elements not fully revealing their capabilities. Our pilot study shows dense connections through concatenation are strong, demonstrating that DenseNets can be revitalized to compete with modern architectures. We methodically refine suboptimal components - architectural adjustments, block redesign, and improved training recipes towards widening DenseNets and boosting memory efficiency while keeping concatenation shortcuts. Our models, employing simple architectural elements, ultimately surpass Swin Transformer, ConvNeXt, and DeiT-III - key architectures in the residual learning lineage. Furthermore, our models exhibit near state-of-the-art performance on ImageNet-1K, competing with the very recent models and downstream tasks, ADE20k semantic segmentation, and COCO object detection/instance segmentation. Finally, we provide empirical analyses that uncover the merits of the concatenation over additive shortcuts, steering a renewed preference towards DenseNet-style designs. Our code is available at https://github.com/naver-ai/rdnet.","url_abs":"https://arxiv.org/abs/2403.19588v2","url_pdf":"https://arxiv.org/pdf/2403.19588v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"densenets-reloaded-paradigm-shift-beyond","repo_url":"https://github.com/huggingface/pytorch-image-models","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"densenets-reloaded-paradigm-shift-beyond","repo_url":"https://github.com/naver-ai/rdnet","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"densenets-reloaded-paradigm-shift-beyond","repo_url":"https://gitlab.com/birder/birder","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"fine-grained-image-classification","task_name":"Fine-Grained Image Classification"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"instance-segmentation","task_name":"Instance Segmentation"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"convnext","method_name":"ConvNeXt"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"rdnet","method_name":"RDNet"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"stochastic-depth","method_name":"Stochastic Depth"},{"method_slug":"swin-transformer","method_name":"Swin Transformer"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[{"slug":"rdnet","name":"RDNet","full_name":"RDNet"}],"results":[{"leaderboard":"/sota/fine-grained-image-classification-on-stanford","task":"Fine-Grained Image Classification","dataset":"Stanford Cars","model":"RDNet-S (224 res, IN-1K pretrained)","rank_in_archive_order":51,"of":83,"metrics":{"Accuracy":"94.2%","FLOPS":"8.7G","PARAMS":"50M"},"uses_additional_data":false},{"leaderboard":"/sota/fine-grained-image-classification-on-stanford","task":"Fine-Grained Image Classification","dataset":"Stanford Cars","model":"RDNet-L (224 res, IN-1K pretrained)","rank_in_archive_order":52,"of":83,"metrics":{"Accuracy":"94.2%","FLOPS":"34.7G","PARAMS":"186M"},"uses_additional_data":false},{"leaderboard":"/sota/fine-grained-image-classification-on-stanford","task":"Fine-Grained Image Classification","dataset":"Stanford Cars","model":"RDNet-B (224 res, IN-1K pretrained)","rank_in_archive_order":53,"of":83,"metrics":{"Accuracy":"94.1%","FLOPS":"15.4G","PARAMS":"87M"},"uses_additional_data":false},{"leaderboard":"/sota/fine-grained-image-classification-on-stanford","task":"Fine-Grained Image Classification","dataset":"Stanford Cars","model":"RDNet-T (224 res, IN-1K pretrained)","rank_in_archive_order":59,"of":83,"metrics":{"Accuracy":"93.9%","FLOPS":"5.0G","PARAMS":"24M"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-cifar-10","task":"Image Classification","dataset":"CIFAR-10","model":"RDNet-L (224 res, IN-1K pretrained)","rank_in_archive_order":8,"of":265,"metrics":{"Percentage correct":"99.31"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-cifar-10","task":"Image Classification","dataset":"CIFAR-10","model":"RDNet-B (224 res, IN-1K pretrained)","rank_in_archive_order":9,"of":265,"metrics":{"Percentage correct":"99.31"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-cifar-10","task":"Image Classification","dataset":"CIFAR-10","model":"RDNet-T (224 res, IN-1K pretrained)","rank_in_archive_order":28,"of":265,"metrics":{"Percentage correct":"98.88"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"RDNet-L (384 res)","rank_in_archive_order":197,"of":1060,"metrics":{"GFLOPs":"34.7","Number of params":"186M","Top 1 Accuracy":"85.8%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"RDNet-L","rank_in_archive_order":292,"of":1060,"metrics":{"GFLOPs":"34.7","Number of params":"186M","Top 1 Accuracy":"84.8%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"RDNet-B","rank_in_archive_order":320,"of":1060,"metrics":{"GFLOPs":"15.4","Number of params":"87M","Top 1 Accuracy":"84.4%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"RDNet-S","rank_in_archive_order":395,"of":1060,"metrics":{"GFLOPs":"8.7","Number of params":"50M","Top 1 Accuracy":"83.7%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"RDNet-T","rank_in_archive_order":496,"of":1060,"metrics":{"GFLOPs":"5.0","Number of params":"24M","Top 1 Accuracy":"82.8%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-inaturalist-2018","task":"Image Classification","dataset":"iNaturalist 2018","model":"RDNet-L (224 res, IN-1K pretrained)","rank_in_archive_order":12,"of":60,"metrics":{"Number of params":"186M","Top-1 Accuracy":"81.8%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-inaturalist-2018","task":"Image Classification","dataset":"iNaturalist 2018","model":"RDNet-B (224 res, IN-1K pretrained)","rank_in_archive_order":16,"of":60,"metrics":{"Number of params":"87M","Top-1 Accuracy":"80.5"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-inaturalist-2018","task":"Image Classification","dataset":"iNaturalist 2018","model":"RDNet-S (224 res, IN-1K pretrained)","rank_in_archive_order":20,"of":60,"metrics":{"Number of params":"50M","Top-1 Accuracy":"79.1"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-inaturalist-2018","task":"Image Classification","dataset":"iNaturalist 2018","model":"RDNet-T (224 res, IN-1K pretrained)","rank_in_archive_order":24,"of":60,"metrics":{"Number of params":"24M","Top-1 Accuracy":"77.0"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-inaturalist-2019","task":"Image Classification","dataset":"iNaturalist 2019","model":"RDNet-L (224 res, IN-1K pretrained)","rank_in_archive_order":5,"of":22,"metrics":{"Number of params":"186M","Top-1 Accuracy":"83.7"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-inaturalist-2019","task":"Image Classification","dataset":"iNaturalist 2019","model":"RDNet-B (224 res, IN-1K pretrained)","rank_in_archive_order":6,"of":22,"metrics":{"Number of params":"87M","Top-1 Accuracy":"83.5"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-inaturalist-2019","task":"Image Classification","dataset":"iNaturalist 2019","model":"RDNet-S (224 res, IN-1K pretrained)","rank_in_archive_order":7,"of":22,"metrics":{"Number of params":"50M","Top-1 Accuracy":"82.9"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-inaturalist-2019","task":"Image Classification","dataset":"iNaturalist 2019","model":"RDNet-T (224 res, IN-1K pretrained)","rank_in_archive_order":11,"of":22,"metrics":{"Number of params":"24M","Top-1 Accuracy":"81.2"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.19588","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.19588"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/naver-ai/rdnet","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_honours":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"410bffb8a07b52e3","entry":"count_params","repo":"naver-ai/rdnet","repo_kind":"official","path":"benchmark.py","file_url":"https://github.com/naver-ai/rdnet/blob/HEAD/benchmark.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":2,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"410bffb8a07b52e3"}},{"code_sha256_prefix":"50d653ee6a59955c","entry":"cuda_timestamp","repo":"naver-ai/rdnet","repo_kind":"official","path":"benchmark.py","file_url":"https://github.com/naver-ai/rdnet/blob/HEAD/benchmark.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"50d653ee6a59955c"}},{"code_sha256_prefix":"475160c5a1a1225a","entry":"timestamp","repo":"naver-ai/rdnet","repo_kind":"official","path":"benchmark.py","file_url":"https://github.com/naver-ai/rdnet/blob/HEAD/benchmark.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"475160c5a1a1225a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}