{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/overinterpretation-reveals-image","title":"Overinterpretation reveals image classification model pathologies","arxiv_id":"2003.08907","date":"2020-03-19","proceeding":"NeurIPS 2021 12","authors":["Brandon Carter","Siddhartha Jain","Jonas Mueller","David Gifford"],"abstract":"Image classifiers are typically scored on their test set accuracy, but high accuracy can mask a subtle type of model failure. We find that high scoring convolutional neural networks (CNNs) on popular benchmarks exhibit troubling pathologies that allow them to display high accuracy even in the absence of semantically salient features. When a model provides a high-confidence decision without salient supporting input features, we say the classifier has overinterpreted its input, finding too much class-evidence in patterns that appear nonsensical to humans. Here, we demonstrate that neural networks trained on CIFAR-10 and ImageNet suffer from overinterpretation, and we find models on CIFAR-10 make confident predictions even when 95% of input images are masked and humans cannot discern salient features in the remaining pixel-subsets. We introduce Batched Gradient SIS, a new method for discovering sufficient input subsets for complex datasets, and use this method to show the sufficiency of border pixels in ImageNet for training and testing. Although these patterns portend potential model fragility in real-world deployment, they are in fact valid statistical patterns of the benchmark that alone suffice to attain high test accuracy. Unlike adversarial examples, overinterpretation relies upon unmodified image pixels. We find ensembling and input dropout can each help mitigate overinterpretation.","url_abs":"https://arxiv.org/abs/2003.08907v3","url_pdf":"https://arxiv.org/pdf/2003.08907v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"overinterpretation-reveals-image","repo_url":"https://github.com/gifford-lab/overinterpretation","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"overinterpretation-reveals-image","repo_url":"https://github.com/b-carter/overinterpretation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"image-classification","task_name":"image-classification"},{"task_slug":"model","task_name":"model"},{"task_slug":null,"task_name":"valid"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2003.08907","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2003.08907"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/gifford-lab/overinterpretation","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/b-carter/overinterpretation","reach":null}],"summary":{"ran":1,"ran_draft_wrong":4,"ran_fixture":3,"ran_honours":2,"unverified":2},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1},"listed":{"samples":9,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2b3638761f79af0a","entry":"SISResult","repo":"b-carter/overinterpretation","repo_kind":"listed","path":"sufficient_input_subsets/sis.py","file_url":"https://github.com/b-carter/overinterpretation/blob/HEAD/sufficient_input_subsets/sis.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2b3638761f79af0a"}},{"code_sha256_prefix":"2da81307bc709549","entry":"_backselect","repo":"b-carter/overinterpretation","repo_kind":"listed","path":"sufficient_input_subsets/sis.py","file_url":"https://github.com/b-carter/overinterpretation/blob/HEAD/sufficient_input_subsets/sis.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2da81307bc709549"}},{"code_sha256_prefix":"1d33e984516a373d","entry":"_find_sis_from_backselect","repo":"b-carter/overinterpretation","repo_kind":"listed","path":"sufficient_input_subsets/sis.py","file_url":"https://github.com/b-carter/overinterpretation/blob/HEAD/sufficient_input_subsets/sis.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1d33e984516a373d"}},{"code_sha256_prefix":"c713a7dcff2871db","entry":"_produce_next_masks","repo":"b-carter/overinterpretation","repo_kind":"listed","path":"sufficient_input_subsets/sis.py","file_url":"https://github.com/b-carter/overinterpretation/blob/HEAD/sufficient_input_subsets/sis.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c713a7dcff2871db"}},{"code_sha256_prefix":"7f24f8f5f6c29543","entry":"_transform_index_array_into_indexer","repo":"b-carter/overinterpretation","repo_kind":"listed","path":"sufficient_input_subsets/sis.py","file_url":"https://github.com/b-carter/overinterpretation/blob/HEAD/sufficient_input_subsets/sis.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7f24f8f5f6c29543"}},{"code_sha256_prefix":"c1c5f65dfa8444b9","entry":"_transform_next_masks_index_array_into_tuple","repo":"b-carter/overinterpretation","repo_kind":"listed","path":"sufficient_input_subsets/sis.py","file_url":"https://github.com/b-carter/overinterpretation/blob/HEAD/sufficient_input_subsets/sis.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c1c5f65dfa8444b9"}},{"code_sha256_prefix":"e9cb5323eca9c3e4","entry":"make_empty_boolean_mask","repo":"gifford-lab/overinterpretation","repo_kind":"official","path":"sufficient_input_subsets/sis.py","file_url":"https://github.com/gifford-lab/overinterpretation/blob/HEAD/sufficient_input_subsets/sis.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e9cb5323eca9c3e4"}},{"code_sha256_prefix":"0a0908809e1bc3b9","entry":"make_empty_boolean_mask","repo":"b-carter/overinterpretation","repo_kind":"listed","path":"sufficient_input_subsets/sis.py","file_url":"https://github.com/b-carter/overinterpretation/blob/HEAD/sufficient_input_subsets/sis.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0a0908809e1bc3b9"}},{"code_sha256_prefix":"0ac134043bf4c5a5","entry":"make_empty_boolean_mask_broadcast_over_axis","repo":"gifford-lab/overinterpretation","repo_kind":"official","path":"sufficient_input_subsets/sis.py","file_url":"https://github.com/gifford-lab/overinterpretation/blob/HEAD/sufficient_input_subsets/sis.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0ac134043bf4c5a5"}},{"code_sha256_prefix":"34fc1f71564be44f","entry":"produce_masked_inputs","repo":"b-carter/overinterpretation","repo_kind":"listed","path":"sufficient_input_subsets/sis.py","file_url":"https://github.com/b-carter/overinterpretation/blob/HEAD/sufficient_input_subsets/sis.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"34fc1f71564be44f"}},{"code_sha256_prefix":"8fe1a968dd21cb11","entry":"find_sis","repo":"b-carter/overinterpretation","repo_kind":"listed","path":"sufficient_input_subsets/sis.py","file_url":"https://github.com/b-carter/overinterpretation/blob/HEAD/sufficient_input_subsets/sis.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8fe1a968dd21cb11"}},{"code_sha256_prefix":"f25a46ac3ec9a39c","entry":"produce_masked_inputs","repo":"gifford-lab/overinterpretation","repo_kind":"official","path":"sufficient_input_subsets/sis.py","file_url":"https://github.com/gifford-lab/overinterpretation/blob/HEAD/sufficient_input_subsets/sis.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f25a46ac3ec9a39c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}