{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/systematic-generalization-what-is-required","title":"Systematic Generalization: What Is Required and Can It Be Learned?","arxiv_id":"1811.12889","date":"2018-11-30","proceeding":"ICLR 2019 5","authors":["Dzmitry Bahdanau","Shikhar Murty","Michael Noukhovitch","Thien Huu Nguyen","Harm de Vries","Aaron Courville"],"abstract":"Numerous models for grounded language understanding have been recently\nproposed, including (i) generic models that can be easily adapted to any given\ntask and (ii) intuitively appealing modular models that require background\nknowledge to be instantiated. We compare both types of models in how much they\nlend themselves to a particular form of systematic generalization. Using a\nsynthetic VQA test, we evaluate which models are capable of reasoning about all\npossible object pairs after training on only a small subset of them. Our\nfindings show that the generalization of modular models is much more systematic\nand that it is highly sensitive to the module layout, i.e. to how exactly the\nmodules are connected. We furthermore investigate if modular models that\ngeneralize well could be made more end-to-end by learning their layout and\nparametrization. We find that end-to-end methods from prior work often learn\ninappropriate layouts or parametrizations that do not facilitate systematic\ngeneralization. Our results suggest that, in addition to modularity, systematic\ngeneralization in language understanding may require explicit regularizers or\npriors.","url_abs":"http://arxiv.org/abs/1811.12889v3","url_pdf":"http://arxiv.org/pdf/1811.12889v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"systematic-generalization-what-is-required","repo_url":"https://github.com/rizar/systematic-generalization-sqoop","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"systematic-generalization-what-is-required","repo_url":"https://github.com/vanessadamario/understanding_reasoning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"systematic-generalization","task_name":"Systematic Generalization"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1811.12889","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1811.12889"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rizar/systematic-generalization-sqoop","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vanessadamario/understanding_reasoning","reach":null}],"summary":{"ran_draft_wrong":3,"ran_honours":2},"by_repo_kind":{},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"3fcd91b7695dd7e6","entry":"get_index","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"3fcd91b7695dd7e6"}},{"code_sha256_prefix":"08cd6598ccb1e857","entry":"one_or_list","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"08cd6598ccb1e857"}},{"code_sha256_prefix":"e7d281257c225992","entry":"parse_float_list","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"e7d281257c225992"}},{"code_sha256_prefix":"db09fab2902ef1f9","entry":"parse_int_list","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"db09fab2902ef1f9"}},{"code_sha256_prefix":"4d6893e38a989635","entry":"save_grad","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"4d6893e38a989635"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}