{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/can-cnns-be-more-robust-than-transformers","title":"Can CNNs Be More Robust Than Transformers?","arxiv_id":"2206.03452","date":"2022-06-07","proceeding":null,"authors":["Zeyu Wang","Yutong Bai","Yuyin Zhou","Cihang Xie"],"abstract":"The recent success of Vision Transformers is shaking the long dominance of Convolutional Neural Networks (CNNs) in image recognition for a decade. Specifically, in terms of robustness on out-of-distribution samples, recent research finds that Transformers are inherently more robust than CNNs, regardless of different training setups. Moreover, it is believed that such superiority of Transformers should largely be credited to their self-attention-like architectures per se. In this paper, we question that belief by closely examining the design of Transformers. Our findings lead to three highly effective architecture designs for boosting robustness, yet simple enough to be implemented in several lines of code, namely a) patchifying input images, b) enlarging kernel size, and c) reducing activation layers and normalization layers. Bringing these components together, we are able to build pure CNN architectures without any attention-like operations that are as robust as, or even more robust than, Transformers. We hope this work can help the community better understand the design of robust neural architectures. The code is publicly available at https://github.com/UCSC-VLAA/RobustCNN.","url_abs":"https://arxiv.org/abs/2206.03452v2","url_pdf":"https://arxiv.org/pdf/2206.03452v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"can-cnns-be-more-robust-than-transformers","repo_url":"https://github.com/ucsc-vlaa/robustcnn","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2206.03452","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.03452"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ucsc-vlaa/robustcnn","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_fixture":1,"ran_honours":1,"unverified":13},"by_repo_kind":{"official":{"samples":15,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c58578dcfe39d5e3","entry":"adapt_input_conv","repo":"ucsc-vlaa/robustcnn","repo_kind":"official","path":"timm/models/helpers.py","file_url":"https://github.com/ucsc-vlaa/robustcnn/blob/HEAD/timm/models/helpers.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c58578dcfe39d5e3"}},{"code_sha256_prefix":"9b02c1068795fd5b","entry":"get_padding","repo":"ucsc-vlaa/robustcnn","repo_kind":"official","path":"timm/models/robust_resnet.py","file_url":"https://github.com/ucsc-vlaa/robustcnn/blob/HEAD/timm/models/robust_resnet.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9b02c1068795fd5b"}},{"code_sha256_prefix":"983c6a8ed9b9f1e3","entry":"create_model","repo":"ucsc-vlaa/robustcnn","repo_kind":"official","path":"timm/models/factory.py","file_url":"https://github.com/ucsc-vlaa/robustcnn/blob/HEAD/timm/models/factory.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"983c6a8ed9b9f1e3"}},{"code_sha256_prefix":"32933dcf801320f2","entry":"download_cached_file","repo":"ucsc-vlaa/robustcnn","repo_kind":"official","path":"timm/models/hub.py","file_url":"https://github.com/ucsc-vlaa/robustcnn/blob/HEAD/timm/models/hub.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"32933dcf801320f2"}},{"code_sha256_prefix":"9f91aef11975b7e7","entry":"downsample_avg","repo":"ucsc-vlaa/robustcnn","repo_kind":"official","path":"timm/models/robust_resnet.py","file_url":"https://github.com/ucsc-vlaa/robustcnn/blob/HEAD/timm/models/robust_resnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9f91aef11975b7e7"}},{"code_sha256_prefix":"ae25309d30fc2b83","entry":"downsample_conv","repo":"ucsc-vlaa/robustcnn","repo_kind":"official","path":"timm/models/robust_resnet.py","file_url":"https://github.com/ucsc-vlaa/robustcnn/blob/HEAD/timm/models/robust_resnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ae25309d30fc2b83"}},{"code_sha256_prefix":"da3ccccf419f89e7","entry":"get_cache_dir","repo":"ucsc-vlaa/robustcnn","repo_kind":"official","path":"timm/models/hub.py","file_url":"https://github.com/ucsc-vlaa/robustcnn/blob/HEAD/timm/models/hub.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"da3ccccf419f89e7"}},{"code_sha256_prefix":"4169528fccdf4859","entry":"has_hf_hub","repo":"ucsc-vlaa/robustcnn","repo_kind":"official","path":"timm/models/hub.py","file_url":"https://github.com/ucsc-vlaa/robustcnn/blob/HEAD/timm/models/hub.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4169528fccdf4859"}},{"code_sha256_prefix":"cd2a47ded9dd84cf","entry":"is_model","repo":"ucsc-vlaa/robustcnn","repo_kind":"official","path":"timm/models/registry.py","file_url":"https://github.com/ucsc-vlaa/robustcnn/blob/HEAD/timm/models/registry.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cd2a47ded9dd84cf"}},{"code_sha256_prefix":"d14a6ca5ff9d530e","entry":"list_models","repo":"ucsc-vlaa/robustcnn","repo_kind":"official","path":"timm/models/registry.py","file_url":"https://github.com/ucsc-vlaa/robustcnn/blob/HEAD/timm/models/registry.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d14a6ca5ff9d530e"}},{"code_sha256_prefix":"a5729ac034a0df02","entry":"load_state_dict","repo":"ucsc-vlaa/robustcnn","repo_kind":"official","path":"timm/models/helpers.py","file_url":"https://github.com/ucsc-vlaa/robustcnn/blob/HEAD/timm/models/helpers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a5729ac034a0df02"}},{"code_sha256_prefix":"e13039fb1c347452","entry":"register_model","repo":"ucsc-vlaa/robustcnn","repo_kind":"official","path":"timm/models/registry.py","file_url":"https://github.com/ucsc-vlaa/robustcnn/blob/HEAD/timm/models/registry.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e13039fb1c347452"}},{"code_sha256_prefix":"102b75f3481035d1","entry":"resume_checkpoint","repo":"ucsc-vlaa/robustcnn","repo_kind":"official","path":"timm/models/helpers.py","file_url":"https://github.com/ucsc-vlaa/robustcnn/blob/HEAD/timm/models/helpers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"102b75f3481035d1"}},{"code_sha256_prefix":"dcad83d71e6ab634","entry":"safe_model_name","repo":"ucsc-vlaa/robustcnn","repo_kind":"official","path":"timm/models/factory.py","file_url":"https://github.com/ucsc-vlaa/robustcnn/blob/HEAD/timm/models/factory.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dcad83d71e6ab634"}},{"code_sha256_prefix":"8b170a922ca9e2f3","entry":"split_model_name","repo":"ucsc-vlaa/robustcnn","repo_kind":"official","path":"timm/models/factory.py","file_url":"https://github.com/ucsc-vlaa/robustcnn/blob/HEAD/timm/models/factory.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8b170a922ca9e2f3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}