{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/are-vision-transformers-more-data-hungry-than-1","title":"Are Vision Transformers More Data Hungry Than Newborn Visual Systems?","arxiv_id":"2312.02843","date":"2023-12-05","proceeding":"NeurIPS 2023 11","authors":["Lalit Pandey","Samantha M. W. Wood","Justin N. Wood"],"abstract":"Vision transformers (ViTs) are top performing models on many computer vision benchmarks and can accurately predict human behavior on object recognition tasks. However, researchers question the value of using ViTs as models of biological learning because ViTs are thought to be more data hungry than brains, with ViTs requiring more training data to reach similar levels of performance. To test this assumption, we directly compared the learning abilities of ViTs and animals, by performing parallel controlled rearing experiments on ViTs and newborn chicks. We first raised chicks in impoverished visual environments containing a single object, then simulated the training data available in those environments by building virtual animal chambers in a video game engine. We recorded the first-person images acquired by agents moving through the virtual chambers and used those images to train self supervised ViTs that leverage time as a teaching signal, akin to biological visual systems. When ViTs were trained through the eyes of newborn chicks, the ViTs solved the same view invariant object recognition tasks as the chicks. Thus, ViTs were not more data hungry than newborn visual systems: both learned view invariant object representations in impoverished visual environments. The flexible and generic attention based learning mechanism in ViTs combined with the embodied data streams available to newborn animals appears sufficient to drive the development of animal-like object recognition.","url_abs":"https://arxiv.org/abs/2312.02843v1","url_pdf":"https://arxiv.org/pdf/2312.02843v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"are-vision-transformers-more-data-hungry-than-1","repo_url":"https://github.com/buildingamind/vit-cot","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"object","task_name":"Object"},{"task_slug":"object-recognition","task_name":"Object Recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2312.02843","atlas_url":"https://app.syntology.ai/?focus=2312.02843","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.02843"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/buildingamind/ViT-CoT","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/buildingamind/vit-cot","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"ran_honours":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8dd8c7db24c1a794","entry":"create_decoder","repo":"buildingamind/vit-cot","repo_kind":"official","path":"models/common.py","file_url":"https://github.com/buildingamind/vit-cot/blob/HEAD/models/common.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8dd8c7db24c1a794"}},{"code_sha256_prefix":"b4b12ef2736a3dae","entry":"create_encoder","repo":"buildingamind/vit-cot","repo_kind":"official","path":"models/common.py","file_url":"https://github.com/buildingamind/vit-cot/blob/HEAD/models/common.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b4b12ef2736a3dae"}},{"code_sha256_prefix":"7bba2e94d7262551","entry":"get_model_output_size","repo":"buildingamind/ViT-CoT","repo_kind":"official","path":"scripts/evaluate.py","file_url":"https://github.com/buildingamind/ViT-CoT/blob/HEAD/scripts/evaluate.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7bba2e94d7262551"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}