{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/revealing-the-dark-secrets-of-extremely-large","title":"Revealing the Dark Secrets of Extremely Large Kernel ConvNets on Robustness","arxiv_id":"2407.08972","date":"2024-07-12","proceeding":null,"authors":["Honghao Chen","Yurong Zhang","Xiaokun Feng","Xiangxiang Chu","Kaiqi Huang"],"abstract":"Robustness is a vital aspect to consider when deploying deep learning models into the wild. Numerous studies have been dedicated to the study of the robustness of vision transformers (ViTs), which have dominated as the mainstream backbone choice for vision tasks since the dawn of 2020s. Recently, some large kernel convnets make a comeback with impressive performance and efficiency. However, it still remains unclear whether large kernel networks are robust and the attribution of their robustness. In this paper, we first conduct a comprehensive evaluation of large kernel convnets' robustness and their differences from typical small kernel counterparts and ViTs on six diverse robustness benchmark datasets. Then to analyze the underlying factors behind their strong robustness, we design experiments from both quantitative and qualitative perspectives to reveal large kernel convnets' intriguing properties that are completely different from typical convnets. Our experiments demonstrate for the first time that pure CNNs can achieve exceptional robustness comparable or even superior to that of ViTs. Our analysis on occlusion invariance, kernel attention patterns and frequency characteristics provide novel insights into the source of robustness.","url_abs":"https://arxiv.org/abs/2407.08972v1","url_pdf":"https://arxiv.org/pdf/2407.08972v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"revealing-the-dark-secrets-of-extremely-large","repo_url":"https://github.com/lauch1ng/lkrobust","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2407.08972","atlas_url":"https://app.syntology.ai/?focus=2407.08972","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.08972"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lauch1ng/lkrobust","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":4,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9d57e8d027d73f7d","entry":"clean_accuracy","repo":"lauch1ng/lkrobust","repo_kind":"official","path":"adversarial_attack/utils.py","file_url":"https://github.com/lauch1ng/lkrobust/blob/HEAD/adversarial_attack/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9d57e8d027d73f7d"}},{"code_sha256_prefix":"b4613da968a83fc8","entry":"get_bn","repo":"lauch1ng/lkrobust","repo_kind":"official","path":"adversarial_attack/replknet.py","file_url":"https://github.com/lauch1ng/lkrobust/blob/HEAD/adversarial_attack/replknet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b4613da968a83fc8"}},{"code_sha256_prefix":"811ac4191033004e","entry":"get_imagenet_data","repo":"lauch1ng/lkrobust","repo_kind":"official","path":"adversarial_attack/utils.py","file_url":"https://github.com/lauch1ng/lkrobust/blob/HEAD/adversarial_attack/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"811ac4191033004e"}},{"code_sha256_prefix":"317cf3d87d623d14","entry":"get_pred","repo":"lauch1ng/lkrobust","repo_kind":"official","path":"adversarial_attack/utils.py","file_url":"https://github.com/lauch1ng/lkrobust/blob/HEAD/adversarial_attack/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"317cf3d87d623d14"}},{"code_sha256_prefix":"b700fb0a3496b255","entry":"cosine_scheduler","repo":"lauch1ng/lkrobust","repo_kind":"official","path":"adversarial_attack/convnext_utils.py","file_url":"https://github.com/lauch1ng/lkrobust/blob/HEAD/adversarial_attack/convnext_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b700fb0a3496b255"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}