{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/improving-robustness-against-common","title":"Improving robustness against common corruptions by covariate shift adaptation","arxiv_id":"2006.16971","date":"2020-06-30","proceeding":"NeurIPS 2020 12","authors":["Steffen Schneider","Evgenia Rusak","Luisa Eck","Oliver Bringmann","Wieland Brendel","Matthias Bethge"],"abstract":"Today's state-of-the-art machine vision models are vulnerable to image corruptions like blurring or compression artefacts, limiting their performance in many real-world applications. We here argue that popular benchmarks to measure model robustness against common corruptions (like ImageNet-C) underestimate model robustness in many (but not all) application scenarios. The key insight is that in many scenarios, multiple unlabeled examples of the corruptions are available and can be used for unsupervised online adaptation. Replacing the activation statistics estimated by batch normalization on the training set with the statistics of the corrupted images consistently improves the robustness across 25 different popular computer vision models. Using the corrected statistics, ResNet-50 reaches 62.2% mCE on ImageNet-C compared to 76.7% without adaptation. With the more robust DeepAugment+AugMix model, we improve the state of the art achieved by a ResNet50 model up to date from 53.6% mCE to 45.4% mCE. Even adapting to a single sample improves robustness for the ResNet-50 and AugMix models, and 32 samples are sufficient to improve the current state of the art for a ResNet-50 architecture. We argue that results with adapted statistics should be included whenever reporting scores in corruption benchmarks and other out-of-distribution generalization settings.","url_abs":"https://arxiv.org/abs/2006.16971v2","url_pdf":"https://arxiv.org/pdf/2006.16971v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"improving-robustness-against-common","repo_url":"https://github.com/bethgelab/robustness","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"improving-robustness-against-common","repo_url":"https://github.com/Claydon-Wang/OFTTA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"domain-generalization","task_name":"Domain Generalization"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"out-of-distribution-generalization","task_name":"Out-of-Distribution Generalization"},{"task_slug":"unsupervised-domain-adaptation","task_name":"Unsupervised Domain Adaptation"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"augmix","method_name":"AugMix"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-classification-on-objectnet","task":"Image Classification","dataset":"ObjectNet","model":"ResNet-50 + GroupNorm","rank_in_archive_order":70,"of":106,"metrics":{"Top-1 Accuracy":"29.2","Top-5 Accuracy":"50.2"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-objectnet","task":"Image Classification","dataset":"ObjectNet","model":"ResNet-50 + RoHL","rank_in_archive_order":71,"of":106,"metrics":{"Top-1 Accuracy":"29.2"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-objectnet","task":"Image Classification","dataset":"ObjectNet","model":"ResNet-50 + FixUp","rank_in_archive_order":74,"of":106,"metrics":{"Top-1 Accuracy":"28.5","Top-5 Accuracy":"48.6"},"uses_additional_data":true},{"leaderboard":"/sota/unsupervised-domain-adaptation-on-imagenet-c","task":"Unsupervised Domain Adaptation","dataset":"ImageNet-C","model":"ResNeXt101+DeepAug+AugMix, BatchNorm Adaptation, full adaptation","rank_in_archive_order":5,"of":16,"metrics":{"mean Corruption Error (mCE)":"38.0"},"uses_additional_data":false},{"leaderboard":"/sota/unsupervised-domain-adaptation-on-imagenet-c","task":"Unsupervised Domain Adaptation","dataset":"ImageNet-C","model":"ResNeXt101+DeepAug+AugMix, BatchNorm Adaptation, 8 samples","rank_in_archive_order":6,"of":16,"metrics":{"mean Corruption Error (mCE)":"40.7"},"uses_additional_data":false},{"leaderboard":"/sota/unsupervised-domain-adaptation-on-imagenet-c","task":"Unsupervised Domain Adaptation","dataset":"ImageNet-C","model":"ResNet50+DeepAug+AugMix, BatchNorm Adaptation, full adaptation","rank_in_archive_order":11,"of":16,"metrics":{"mean Corruption Error (mCE)":"45.4"},"uses_additional_data":false},{"leaderboard":"/sota/unsupervised-domain-adaptation-on-imagenet-c","task":"Unsupervised Domain Adaptation","dataset":"ImageNet-C","model":"ResNet50+DeepAug+AugMix, BatchNorm Adaptation, 8 samples","rank_in_archive_order":12,"of":16,"metrics":{"mean Corruption Error (mCE)":"48.4"},"uses_additional_data":false},{"leaderboard":"/sota/unsupervised-domain-adaptation-on-imagenet-c","task":"Unsupervised Domain Adaptation","dataset":"ImageNet-C","model":"ResNet50 (baseline), BatchNorm Adaptation, full adaptation","rank_in_archive_order":15,"of":16,"metrics":{"mean Corruption Error (mCE)":"62.2"},"uses_additional_data":false},{"leaderboard":"/sota/unsupervised-domain-adaptation-on-imagenet-c","task":"Unsupervised Domain Adaptation","dataset":"ImageNet-C","model":"ResNet50 (baseline), BatchNorm Adaptation, 8 samples","rank_in_archive_order":16,"of":16,"metrics":{"mean Corruption Error (mCE)":"65.0"},"uses_additional_data":false},{"leaderboard":"/sota/unsupervised-domain-adaptation-on-imagenet-r","task":"Unsupervised Domain Adaptation","dataset":"ImageNet-R","model":"ResNeXt101+DeepAug+AugMix, BatchNorm Adaptation,","rank_in_archive_order":4,"of":8,"metrics":{"Top 1 Error":"44.0"},"uses_additional_data":false},{"leaderboard":"/sota/unsupervised-domain-adaptation-on-imagenet-r","task":"Unsupervised Domain Adaptation","dataset":"ImageNet-R","model":"ResNet50+DeepAug+Augmix, BatchNorm adaptation","rank_in_archive_order":5,"of":8,"metrics":{"Top 1 Error":"48.9"},"uses_additional_data":false},{"leaderboard":"/sota/unsupervised-domain-adaptation-on-imagenet-r","task":"Unsupervised Domain Adaptation","dataset":"ImageNet-R","model":"ResNet50, BatchNorm adaptation","rank_in_archive_order":8,"of":8,"metrics":{"Top 1 Error":"59.9"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2006.16971","atlas_url":"https://app.syntology.ai/?focus=2006.16971","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2006.16971"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Claydon-Wang/OFTTA","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bethgelab/robustness","reach":null}],"summary":{"ran_draft_wrong":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ede060eaf2676c57","entry":"adapt_bayesian","repo":"bethgelab/robustness","repo_kind":"official","path":"robusta/batchnorm/bn.py","file_url":"https://github.com/bethgelab/robustness/blob/HEAD/robusta/batchnorm/bn.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ede060eaf2676c57"}},{"code_sha256_prefix":"68723caf24e80266","entry":"adapt_ema","repo":"bethgelab/robustness","repo_kind":"official","path":"robusta/batchnorm/bn.py","file_url":"https://github.com/bethgelab/robustness/blob/HEAD/robusta/batchnorm/bn.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"68723caf24e80266"}},{"code_sha256_prefix":"0c1aa79f9c5ad38c","entry":"adapt_parts","repo":"bethgelab/robustness","repo_kind":"official","path":"robusta/batchnorm/bn.py","file_url":"https://github.com/bethgelab/robustness/blob/HEAD/robusta/batchnorm/bn.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0c1aa79f9c5ad38c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}