{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/removing-batch-normalization-boosts","title":"Removing Batch Normalization Boosts Adversarial Training","arxiv_id":"2207.01156","date":"2022-07-04","proceeding":null,"authors":["Haotao Wang","Aston Zhang","Shuai Zheng","Xingjian Shi","Mu Li","Zhangyang Wang"],"abstract":"Adversarial training (AT) defends deep neural networks against adversarial attacks. One challenge that limits its practical application is the performance degradation on clean samples. A major bottleneck identified by previous works is the widely used batch normalization (BN), which struggles to model the different statistics of clean and adversarial training samples in AT. Although the dominant approach is to extend BN to capture this mixture of distribution, we propose to completely eliminate this bottleneck by removing all BN layers in AT. Our normalizer-free robust training (NoFrost) method extends recent advances in normalizer-free networks to AT for its unexplored advantage on handling the mixture distribution challenge. We show that NoFrost achieves adversarial robustness with only a minor sacrifice on clean sample accuracy. On ImageNet with ResNet50, NoFrost achieves $74.06\\%$ clean accuracy, which drops merely $2.00\\%$ from standard training. In contrast, BN-based AT obtains $59.28\\%$ clean accuracy, suffering a significant $16.78\\%$ drop from standard training. In addition, NoFrost achieves a $23.56\\%$ adversarial robustness against PGD attack, which improves the $13.57\\%$ robustness in BN-based AT. We observe better model smoothness and larger decision margins from NoFrost, which make the models less sensitive to input perturbations and thus more robust. Moreover, when incorporating more data augmentations into NoFrost, it achieves comprehensive robustness against multiple distribution shifts. Code and pre-trained models are public at https://github.com/amazon-research/normalizer-free-robust-training.","url_abs":"https://arxiv.org/abs/2207.01156v1","url_pdf":"https://arxiv.org/pdf/2207.01156v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"removing-batch-normalization-boosts","repo_url":"https://github.com/amazon-research/normalizer-free-robust-training","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"adversarial-robustness","task_name":"Adversarial Robustness"}],"methods":[{"method_slug":"batch-normalization","method_name":"Batch Normalization"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2207.01156","atlas_url":"https://app.syntology.ai/?focus=2207.01156","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2207.01156"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/amazon-research/normalizer-free-robust-training","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":1,"ran":7},"by_repo_kind":{"official":{"samples":8,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8b0e794d4d8f9b13","entry":"default_conv","repo":"amazon-research/normalizer-free-robust-training","repo_kind":"official","path":"deepaug/EDSR_Model/common.py","file_url":"https://github.com/amazon-research/normalizer-free-robust-training/blob/HEAD/deepaug/EDSR_Model/common.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8b0e794d4d8f9b13"}},{"code_sha256_prefix":"36a1160d7f36cda6","entry":"find_classes","repo":"amazon-research/normalizer-free-robust-training","repo_kind":"official","path":"deepaug/CAE_distort_imagenet.py","file_url":"https://github.com/amazon-research/normalizer-free-robust-training/blob/HEAD/deepaug/CAE_distort_imagenet.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"36a1160d7f36cda6"}},{"code_sha256_prefix":"94226f82998bee03","entry":"get_color_distortion","repo":"amazon-research/normalizer-free-robust-training","repo_kind":"official","path":"dataloaders/imagenet.py","file_url":"https://github.com/amazon-research/normalizer-free-robust-training/blob/HEAD/dataloaders/imagenet.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"94226f82998bee03"}},{"code_sha256_prefix":"73681c24fc0f7154","entry":"imagenet_datasets","repo":"amazon-research/normalizer-free-robust-training","repo_kind":"official","path":"dataloaders/imagenet.py","file_url":"https://github.com/amazon-research/normalizer-free-robust-training/blob/HEAD/dataloaders/imagenet.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"73681c24fc0f7154"}},{"code_sha256_prefix":"26ccec159c6509f3","entry":"imagenet_deepaug_dataset","repo":"amazon-research/normalizer-free-robust-training","repo_kind":"official","path":"dataloaders/imagenet.py","file_url":"https://github.com/amazon-research/normalizer-free-robust-training/blob/HEAD/dataloaders/imagenet.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"26ccec159c6509f3"}},{"code_sha256_prefix":"cfb106a12475f94c","entry":"load_ckpt","repo":"amazon-research/normalizer-free-robust-training","repo_kind":"official","path":"utils/utils.py","file_url":"https://github.com/amazon-research/normalizer-free-robust-training/blob/HEAD/utils/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cfb106a12475f94c"}},{"code_sha256_prefix":"e63b113d977c2bc2","entry":"load_ckpt_adv","repo":"amazon-research/normalizer-free-robust-training","repo_kind":"official","path":"utils/utils.py","file_url":"https://github.com/amazon-research/normalizer-free-robust-training/blob/HEAD/utils/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e63b113d977c2bc2"}},{"code_sha256_prefix":"bf9bce3661ad7ca2","entry":"soft_sign","repo":"amazon-research/normalizer-free-robust-training","repo_kind":"official","path":"utils/utils.py","file_url":"https://github.com/amazon-research/normalizer-free-robust-training/blob/HEAD/utils/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bf9bce3661ad7ca2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}