{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bongard-logo-a-new-benchmark-for-human-level","title":"Bongard-LOGO: A New Benchmark for Human-Level Concept Learning and Reasoning","arxiv_id":"2010.00763","date":"2020-10-02","proceeding":"NeurIPS 2020 12","authors":["Weili Nie","Zhiding Yu","Lei Mao","Ankit B. Patel","Yuke Zhu","Animashree Anandkumar"],"abstract":"Humans have an inherent ability to learn novel concepts from only a few samples and generalize these concepts to different situations. Even though today's machine learning models excel with a plethora of training data on standard recognition tasks, a considerable gap exists between machine-level pattern recognition and human-level concept learning. To narrow this gap, the Bongard problems (BPs) were introduced as an inspirational challenge for visual cognition in intelligent systems. Despite new advances in representation learning and learning to learn, BPs remain a daunting challenge for modern AI. Inspired by the original one hundred BPs, we propose a new benchmark Bongard-LOGO for human-level concept learning and reasoning. We develop a program-guided generation technique to produce a large set of human-interpretable visual cognition problems in action-oriented LOGO language. Our benchmark captures three core properties of human cognition: 1) context-dependent perception, in which the same object may have disparate interpretations given different contexts; 2) analogy-making perception, in which some meaningful concepts are traded off for other meaningful concepts; and 3) perception with a few samples but infinite vocabulary. In experiments, we show that the state-of-the-art deep learning methods perform substantially worse than human subjects, implying that they fail to capture core human cognition properties. Finally, we discuss research directions towards a general architecture for visual reasoning to tackle this benchmark.","url_abs":"https://arxiv.org/abs/2010.00763v4","url_pdf":"https://arxiv.org/pdf/2010.00763v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bongard-logo-a-new-benchmark-for-human-level","repo_url":"https://github.com/NVlabs/Bongard-LOGO","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"novel-concepts","task_name":"Novel Concepts"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"visual-reasoning","task_name":"Visual Reasoning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2010.00763","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2010.00763"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/NVlabs/Bongard-LOGO","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":4,"ran_draft_wrong":4,"ran_honours":1,"unverified":5},"by_repo_kind":{"official":{"samples":14,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":13,"samples":[{"code_sha256_prefix":"691b2fde39c5dae5","entry":"computeGramMatrix","repo":"NVlabs/Bongard-LOGO","repo_kind":"official","path":"Bongard-LOGO_Baselines/models/metaOptNet.py","file_url":"https://github.com/NVlabs/Bongard-LOGO/blob/HEAD/Bongard-LOGO_Baselines/models/metaOptNet.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"691b2fde39c5dae5"}},{"code_sha256_prefix":"d9def42110729a85","entry":"conv1x1","repo":"NVlabs/Bongard-LOGO","repo_kind":"official","path":"Bongard-LOGO_Baselines/models/resnet.py","file_url":"https://github.com/NVlabs/Bongard-LOGO/blob/HEAD/Bongard-LOGO_Baselines/models/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"d9def42110729a85"}},{"code_sha256_prefix":"de92832114f71d8c","entry":"conv1x1","repo":"NVlabs/Bongard-LOGO","repo_kind":"official","path":"Bongard-LOGO_Baselines/models/resnet12.py","file_url":"https://github.com/NVlabs/Bongard-LOGO/blob/HEAD/Bongard-LOGO_Baselines/models/resnet12.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"de92832114f71d8c"}},{"code_sha256_prefix":"160bb14bd76201b4","entry":"conv3x3","repo":"NVlabs/Bongard-LOGO","repo_kind":"official","path":"Bongard-LOGO_Baselines/models/resnet.py","file_url":"https://github.com/NVlabs/Bongard-LOGO/blob/HEAD/Bongard-LOGO_Baselines/models/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"160bb14bd76201b4"}},{"code_sha256_prefix":"e5b7c79b0b62af96","entry":"conv3x3","repo":"NVlabs/Bongard-LOGO","repo_kind":"official","path":"Bongard-LOGO_Baselines/models/resnet12.py","file_url":"https://github.com/NVlabs/Bongard-LOGO/blob/HEAD/Bongard-LOGO_Baselines/models/resnet12.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"e5b7c79b0b62af96"}},{"code_sha256_prefix":"9d556d61294bb0a8","entry":"conv_block","repo":"NVlabs/Bongard-LOGO","repo_kind":"official","path":"Bongard-LOGO_Baselines/models/convnet4.py","file_url":"https://github.com/NVlabs/Bongard-LOGO/blob/HEAD/Bongard-LOGO_Baselines/models/convnet4.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"9d556d61294bb0a8"}},{"code_sha256_prefix":"f36ac42575cd46ee","entry":"norm_layer","repo":"NVlabs/Bongard-LOGO","repo_kind":"official","path":"Bongard-LOGO_Baselines/models/resnet12.py","file_url":"https://github.com/NVlabs/Bongard-LOGO/blob/HEAD/Bongard-LOGO_Baselines/models/resnet12.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"f36ac42575cd46ee"}},{"code_sha256_prefix":"44c3aa4c610ea3f7","entry":"register","repo":"NVlabs/Bongard-LOGO","repo_kind":"official","path":"Bongard-LOGO_Baselines/models/models.py","file_url":"https://github.com/NVlabs/Bongard-LOGO/blob/HEAD/Bongard-LOGO_Baselines/models/models.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"44c3aa4c610ea3f7"}},{"code_sha256_prefix":"6aaf07ccda31ac5f","entry":"repackage_hidden","repo":"NVlabs/Bongard-LOGO","repo_kind":"official","path":"Bongard-LOGO_Baselines/models/recurrent_model.py","file_url":"https://github.com/NVlabs/Bongard-LOGO/blob/HEAD/Bongard-LOGO_Baselines/models/recurrent_model.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6aaf07ccda31ac5f"}},{"code_sha256_prefix":"7154d9592085167c","entry":"ProtoNetHead","repo":"NVlabs/Bongard-LOGO","repo_kind":"official","path":"Bongard-LOGO_Baselines/models/metaOptNet.py","file_url":"https://github.com/NVlabs/Bongard-LOGO/blob/HEAD/Bongard-LOGO_Baselines/models/metaOptNet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"7154d9592085167c"}},{"code_sha256_prefix":"660433bc1546c183","entry":"get_subdict","repo":"NVlabs/Bongard-LOGO","repo_kind":"official","path":"Bongard-LOGO_Baselines/models/maml.py","file_url":"https://github.com/NVlabs/Bongard-LOGO/blob/HEAD/Bongard-LOGO_Baselines/models/maml.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"660433bc1546c183"}},{"code_sha256_prefix":"25ae334afa9e8b9f","entry":"gradient_update_parameters","repo":"NVlabs/Bongard-LOGO","repo_kind":"official","path":"Bongard-LOGO_Baselines/models/maml.py","file_url":"https://github.com/NVlabs/Bongard-LOGO/blob/HEAD/Bongard-LOGO_Baselines/models/maml.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"25ae334afa9e8b9f"}},{"code_sha256_prefix":"863137f968d165f1","entry":"load","repo":"NVlabs/Bongard-LOGO","repo_kind":"official","path":"Bongard-LOGO_Baselines/models/models.py","file_url":"https://github.com/NVlabs/Bongard-LOGO/blob/HEAD/Bongard-LOGO_Baselines/models/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"863137f968d165f1"}},{"code_sha256_prefix":"220503b76dce8b41","entry":"make","repo":"NVlabs/Bongard-LOGO","repo_kind":"official","path":"Bongard-LOGO_Baselines/models/models.py","file_url":"https://github.com/NVlabs/Bongard-LOGO/blob/HEAD/Bongard-LOGO_Baselines/models/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"220503b76dce8b41"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}