{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/generalisation-in-humans-and-deep-neural","title":"Generalisation in humans and deep neural networks","arxiv_id":"1808.08750","date":"2018-08-27","proceeding":"NeurIPS 2018 12","authors":["Robert Geirhos","Carlos R. Medina Temme","Jonas Rauber","Heiko H. Schütt","Matthias Bethge","Felix A. Wichmann"],"abstract":"We compare the robustness of humans and current convolutional deep neural networks (DNNs) on object recognition under twelve different types of image degradations. First, using three well known DNNs (ResNet-152, VGG-19, GoogLeNet) we find the human visual system to be more robust to nearly all of the tested image manipulations, and we observe progressively diverging classification error-patterns between humans and DNNs when the signal gets weaker. Secondly, we show that DNNs trained directly on distorted images consistently surpass human performance on the exact distortion types they were trained on, yet they display extremely poor generalisation abilities when tested on other distortion types. For example, training on salt-and-pepper noise does not imply robustness on uniform white noise and vice versa. Thus, changes in the noise distribution between training and testing constitutes a crucial challenge to deep learning vision systems that can be systematically addressed in a lifelong machine learning approach. Our new dataset consisting of 83K carefully measured human psychophysical trials provide a useful reference for lifelong robustness against image degradations set by the human visual system.","url_abs":"https://arxiv.org/abs/1808.08750v3","url_pdf":"https://arxiv.org/pdf/1808.08750v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"generalisation-in-humans-and-deep-neural","repo_url":"https://github.com/rgeirhos/generalisation-humans-DNNs","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"generalisation-in-humans-and-deep-neural","repo_url":"https://github.com/rgeirhos/texture-vs-shape","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"object-recognition","task_name":"Object Recognition"}],"methods":[{"method_slug":"vgg-19","method_name":"VGG-19"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1808.08750","atlas_url":"https://app.syntology.ai/?focus=1808.08750","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1808.08750"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rgeirhos/texture-vs-shape","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rgeirhos/generalisation-humans-DNNs","reach":null}],"summary":{"ran_fixture":1,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"e92a4458fd3a9644","entry":"adjust_contrast","repo":"rgeirhos/generalisation-humans-DNNs","repo_kind":"official","path":"code/image_manipulation.py","file_url":"https://github.com/rgeirhos/generalisation-humans-DNNs/blob/HEAD/code/image_manipulation.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"e92a4458fd3a9644"}},{"code_sha256_prefix":"575ec46a39a8e8a1","entry":"get_WNID_from_index","repo":"rgeirhos/generalisation-humans-DNNs","repo_kind":"official","path":"code/accuracy_evaluation/evaluate.py","file_url":"https://github.com/rgeirhos/generalisation-humans-DNNs/blob/HEAD/code/accuracy_evaluation/evaluate.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"575ec46a39a8e8a1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}