{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/harmonizing-the-object-recognition-strategies","title":"Harmonizing the object recognition strategies of deep neural networks with humans","arxiv_id":"2211.04533","date":"2022-11-08","proceeding":null,"authors":["Thomas Fel","Ivan Felipe","Drew Linsley","Thomas Serre"],"abstract":"The many successes of deep neural networks (DNNs) over the past decade have largely been driven by computational scale rather than insights from biological intelligence. Here, we explore if these trends have also carried concomitant improvements in explaining the visual strategies humans rely on for object recognition. We do this by comparing two related but distinct properties of visual strategies in humans and DNNs: where they believe important visual features are in images and how they use those features to categorize objects. Across 84 different DNNs trained on ImageNet and three independent datasets measuring the where and the how of human visual strategies for object recognition on those images, we find a systematic trade-off between DNN categorization accuracy and alignment with human visual strategies for object recognition. State-of-the-art DNNs are progressively becoming less aligned with humans as their accuracy improves. We rectify this growing issue with our neural harmonizer: a general-purpose training routine that both aligns DNN and human visual strategies and improves categorization accuracy. Our work represents the first demonstration that the scaling laws that are guiding the design of DNNs today have also produced worse models of human vision. We release our code and data at https://serre-lab.github.io/Harmonization to help the field build more human-like DNNs.","url_abs":"https://arxiv.org/abs/2211.04533v2","url_pdf":"https://arxiv.org/pdf/2211.04533v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"harmonizing-the-object-recognition-strategies","repo_url":"https://github.com/serre-lab/harmonization","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"harmonizing-the-object-recognition-strategies","repo_url":"https://github.com/serre-lab/meta-predictor","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"harmonizing-the-object-recognition-strategies","repo_url":"https://github.com/vicco-group/thingsvision","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"object","task_name":"Object"},{"task_slug":"object-recognition","task_name":"Object Recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2211.04533","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2211.04533"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/tensorflow/tpu","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/serre-lab/harmonization","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/serre-lab/meta-predictor","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vicco-group/thingsvision","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":7},"by_repo_kind":{"listed":{"samples":7,"ran":0,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6e06509da1adcf32","entry":"device_type","repo":"vicco-group/thingsvision","repo_kind":"listed","path":"thingsvision/thingsvision.py","file_url":"https://github.com/vicco-group/thingsvision/blob/HEAD/thingsvision/thingsvision.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6e06509da1adcf32"}},{"code_sha256_prefix":"db2d5a2351ace6a7","entry":"gaussian_blur","repo":"serre-lab/harmonization","repo_kind":"listed","path":"harmonization/common/blur.py","file_url":"https://github.com/serre-lab/harmonization/blob/HEAD/harmonization/common/blur.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"db2d5a2351ace6a7"}},{"code_sha256_prefix":"4f95baae9c000390","entry":"gaussian_kernel","repo":"serre-lab/harmonization","repo_kind":"listed","path":"harmonization/common/blur.py","file_url":"https://github.com/serre-lab/harmonization/blob/HEAD/harmonization/common/blur.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4f95baae9c000390"}},{"code_sha256_prefix":"1c03c112f6bddd71","entry":"harmonizer_loss","repo":"serre-lab/harmonization","repo_kind":"listed","path":"harmonization/training/loss.py","file_url":"https://github.com/serre-lab/harmonization/blob/HEAD/harmonization/training/loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1c03c112f6bddd71"}},{"code_sha256_prefix":"41aa39036593bcbf","entry":"preprocess_input","repo":"serre-lab/harmonization","repo_kind":"listed","path":"harmonization/models/preprocess.py","file_url":"https://github.com/serre-lab/harmonization/blob/HEAD/harmonization/models/preprocess.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"41aa39036593bcbf"}},{"code_sha256_prefix":"ef2a4ddcffa60435","entry":"pyramidal_mse_with_tokens","repo":"serre-lab/harmonization","repo_kind":"listed","path":"harmonization/training/loss.py","file_url":"https://github.com/serre-lab/harmonization/blob/HEAD/harmonization/training/loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ef2a4ddcffa60435"}},{"code_sha256_prefix":"0b5c6594979eb7cf","entry":"standardize_cut","repo":"serre-lab/harmonization","repo_kind":"listed","path":"harmonization/training/loss.py","file_url":"https://github.com/serre-lab/harmonization/blob/HEAD/harmonization/training/loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0b5c6594979eb7cf"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}