{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/battle-of-the-backbones-a-large-scale","title":"Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Tasks","arxiv_id":"2310.19909","date":"2023-10-30","proceeding":"NeurIPS 2023 11","authors":["Micah Goldblum","Hossein Souri","Renkun Ni","Manli Shu","Viraj Prabhu","Gowthami Somepalli","Prithvijit Chattopadhyay","Mark Ibrahim","Adrien Bardes","Judy Hoffman","Rama Chellappa","Andrew Gordon Wilson","Tom Goldstein"],"abstract":"Neural network based computer vision systems are typically built on a backbone, a pretrained or randomly initialized feature extractor. Several years ago, the default option was an ImageNet-trained convolutional neural network. However, the recent past has seen the emergence of countless backbones pretrained using various algorithms and datasets. While this abundance of choice has led to performance increases for a range of systems, it is difficult for practitioners to make informed decisions about which backbone to choose. Battle of the Backbones (BoB) makes this choice easier by benchmarking a diverse suite of pretrained models, including vision-language models, those trained via self-supervised learning, and the Stable Diffusion backbone, across a diverse set of computer vision tasks ranging from classification to object detection to OOD generalization and more. Furthermore, BoB sheds light on promising directions for the research community to advance computer vision by illuminating strengths and weakness of existing approaches through a comprehensive analysis conducted on more than 1500 training runs. While vision transformers (ViTs) and self-supervised learning (SSL) are increasingly popular, we find that convolutional neural networks pretrained in a supervised fashion on large training sets still perform best on most tasks among the models we consider. Moreover, in apples-to-apples comparisons on the same architectures and similarly sized pretraining datasets, we find that SSL backbones are highly competitive, indicating that future works should perform SSL pretraining with advanced architectures and larger pretraining datasets. We release the raw results of our experiments along with code that allows researchers to put their own backbones through the gauntlet here: https://github.com/hsouri/Battle-of-the-Backbones","url_abs":"https://arxiv.org/abs/2310.19909v2","url_pdf":"https://arxiv.org/pdf/2310.19909v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"battle-of-the-backbones-a-large-scale","repo_url":"https://github.com/hsouri/battle-of-the-backbones","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"battle-of-the-backbones-a-large-scale","repo_url":"https://github.com/unveilx/slm-od-ml-comparison","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.19909","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.19909"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/hsouri/Battle-of-the-Backbones","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hsouri/battle-of-the-backbones","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/unveilx/slm-od-ml-comparison","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":6},"by_repo_kind":{"listed":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3f1bb5cc21872995","entry":"create_temp_file","repo":"unveilx/slm-od-ml-comparison","repo_kind":"listed","path":"model/detr/code/common_utils.py","file_url":"https://github.com/unveilx/slm-od-ml-comparison/blob/HEAD/model/detr/code/common_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3f1bb5cc21872995"}},{"code_sha256_prefix":"26aa0a1415a890ac","entry":"get_max_image_size","repo":"unveilx/slm-od-ml-comparison","repo_kind":"listed","path":"model/detr/code/mmdet_mlflow_model_wrapper.py","file_url":"https://github.com/unveilx/slm-od-ml-comparison/blob/HEAD/model/detr/code/mmdet_mlflow_model_wrapper.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"26aa0a1415a890ac"}},{"code_sha256_prefix":"4aa50694be302159","entry":"make_batch","repo":"unveilx/slm-od-ml-comparison","repo_kind":"listed","path":"model/detr/code/mmdet_utils.py","file_url":"https://github.com/unveilx/slm-od-ml-comparison/blob/HEAD/model/detr/code/mmdet_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4aa50694be302159"}},{"code_sha256_prefix":"a0354e67de81dc2d","entry":"pad_img","repo":"unveilx/slm-od-ml-comparison","repo_kind":"listed","path":"model/detr/code/mmdet_utils.py","file_url":"https://github.com/unveilx/slm-od-ml-comparison/blob/HEAD/model/detr/code/mmdet_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a0354e67de81dc2d"}},{"code_sha256_prefix":"25fa358fbfbdb224","entry":"process_image","repo":"unveilx/slm-od-ml-comparison","repo_kind":"listed","path":"model/detr/code/common_utils.py","file_url":"https://github.com/unveilx/slm-od-ml-comparison/blob/HEAD/model/detr/code/common_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"25fa358fbfbdb224"}},{"code_sha256_prefix":"cf2bcc3bf3a5463e","entry":"process_video","repo":"unveilx/slm-od-ml-comparison","repo_kind":"listed","path":"model/detr/code/common_utils.py","file_url":"https://github.com/unveilx/slm-od-ml-comparison/blob/HEAD/model/detr/code/common_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cf2bcc3bf3a5463e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}