{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/can-biases-in-imagenet-models-explain","title":"Can Biases in ImageNet Models Explain Generalization?","arxiv_id":"2404.01509","date":"2024-04-01","proceeding":"CVPR 2024 1","authors":["Paul Gavrikov","Janis Keuper"],"abstract":"The robust generalization of models to rare, in-distribution (ID) samples drawn from the long tail of the training distribution and to out-of-training-distribution (OOD) samples is one of the major challenges of current deep learning methods. For image classification, this manifests in the existence of adversarial attacks, the performance drops on distorted images, and a lack of generalization to concepts such as sketches. The current understanding of generalization in neural networks is very limited, but some biases that differentiate models from human vision have been identified and might be causing these limitations. Consequently, several attempts with varying success have been made to reduce these biases during training to improve generalization. We take a step back and sanity-check these attempts. Fixing the architecture to the well-established ResNet-50, we perform a large-scale study on 48 ImageNet models obtained via different training methods to understand how and if these biases - including shape bias, spectral biases, and critical bands - interact with generalization. Our extensive study results reveal that contrary to previous findings, these biases are insufficient to accurately predict the generalization of a model holistically. We provide access to all checkpoints and evaluation code at https://github.com/paulgavrikov/biases_vs_generalization","url_abs":"https://arxiv.org/abs/2404.01509v1","url_pdf":"https://arxiv.org/pdf/2404.01509v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"can-biases-in-imagenet-models-explain","repo_url":"https://github.com/paulgavrikov/biases_vs_generalization","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"CC-BY-SA-4.0"}}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"vision-transformer","method_name":"Vision Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2404.01509","atlas_url":"https://app.syntology.ai/?focus=2404.01509","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.01509"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/paulgavrikov/biases_vs_generalization","reach":{"status":"ok","spdx":"CC-BY-SA-4.0"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/paulgavrikov/biases_","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}}],"summary":{"ran":6,"unverified":3},"by_repo_kind":{"official":{"samples":9,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"1468a1f57586ad4d","entry":"bandpass_filter","repo":"paulgavrikov/biases_vs_generalization","repo_kind":"official","path":"code/eval_spectral.py","file_url":"https://github.com/paulgavrikov/biases_vs_generalization/blob/HEAD/code/eval_spectral.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"1468a1f57586ad4d"}},{"code_sha256_prefix":"e8501bbc645028fa","entry":"contrast_normalize_np","repo":"paulgavrikov/biases_vs_generalization","repo_kind":"official","path":"code/eval_criticalband.py","file_url":"https://github.com/paulgavrikov/biases_vs_generalization/blob/HEAD/code/eval_criticalband.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"e8501bbc645028fa"}},{"code_sha256_prefix":"71514d366caae3fe","entry":"deque_loader","repo":"paulgavrikov/biases_vs_generalization","repo_kind":"official","path":"code/eval_imagenet_adv.py","file_url":"https://github.com/paulgavrikov/biases_vs_generalization/blob/HEAD/code/eval_imagenet_adv.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"71514d366caae3fe"}},{"code_sha256_prefix":"f96ad5482ab2b9ba","entry":"parse_aa_log","repo":"paulgavrikov/biases_vs_generalization","repo_kind":"official","path":"code/eval_imagenet_adv.py","file_url":"https://github.com/paulgavrikov/biases_vs_generalization/blob/HEAD/code/eval_imagenet_adv.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"f96ad5482ab2b9ba"}},{"code_sha256_prefix":"b1945a1c081f47c5","entry":"r50_tf_to_torch","repo":"paulgavrikov/biases_vs_generalization","repo_kind":"official","path":"code/model_zoo.py","file_url":"https://github.com/paulgavrikov/biases_vs_generalization/blob/HEAD/code/model_zoo.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"b1945a1c081f47c5"}},{"code_sha256_prefix":"54537b5da9539618","entry":"rmspower","repo":"paulgavrikov/biases_vs_generalization","repo_kind":"official","path":"code/eval_criticalband.py","file_url":"https://github.com/paulgavrikov/biases_vs_generalization/blob/HEAD/code/eval_criticalband.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"54537b5da9539618"}},{"code_sha256_prefix":"2c006f49031dde59","entry":"find_run_id_by_name","repo":"paulgavrikov/biases_vs_generalization","repo_kind":"official","path":"code/eval_imagenet_c.py","file_url":"https://github.com/paulgavrikov/biases_vs_generalization/blob/HEAD/code/eval_imagenet_c.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"2c006f49031dde59"}},{"code_sha256_prefix":"461c3ff19b612b52","entry":"get_normalized_model","repo":"paulgavrikov/biases_vs_generalization","repo_kind":"official","path":"code/model_zoo.py","file_url":"https://github.com/paulgavrikov/biases_vs_generalization/blob/HEAD/code/model_zoo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"461c3ff19b612b52"}},{"code_sha256_prefix":"f4f66628816fbe4e","entry":"load_state_dict_from_gdrive","repo":"paulgavrikov/biases_vs_generalization","repo_kind":"official","path":"code/model_zoo.py","file_url":"https://github.com/paulgavrikov/biases_vs_generalization/blob/HEAD/code/model_zoo.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"f4f66628816fbe4e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}