{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/revisiting-the-compositional-generalization-1","title":"Revisiting the Compositional Generalization Abilities of Neural Sequence Models","arxiv_id":"2203.07402","date":"2022-03-14","proceeding":"ACL 2022 5","authors":["Arkil Patel","Satwik Bhattamishra","Phil Blunsom","Navin Goyal"],"abstract":"Compositional generalization is a fundamental trait in humans, allowing us to effortlessly combine known phrases to form novel sentences. Recent works have claimed that standard seq-to-seq models severely lack the ability to compositionally generalize. In this paper, we focus on one-shot primitive generalization as introduced by the popular SCAN benchmark. We demonstrate that modifying the training distribution in simple and intuitive ways enables standard seq-to-seq models to achieve near-perfect generalization performance, thereby showing that their compositional generalization abilities were previously underestimated. We perform detailed empirical analysis of this phenomenon. Our results indicate that the generalization performance of models is highly sensitive to the characteristics of the training data which should be carefully considered while designing such benchmarks in future.","url_abs":"https://arxiv.org/abs/2203.07402v1","url_pdf":"https://arxiv.org/pdf/2203.07402v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"revisiting-the-compositional-generalization-1","repo_url":"https://github.com/arkilpatel/compositional-generalization-seq2seq","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2203.07402","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.07402"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/arkilpatel/compositional-generalization-seq2seq","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":2},"by_repo_kind":{"official":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e377a9696d9833a7","entry":"bert_sim","repo":"arkilpatel/compositional-generalization-seq2seq","repo_kind":"official","path":"code/lstm/src/confidence_estimation.py","file_url":"https://github.com/arkilpatel/compositional-generalization-seq2seq/blob/HEAD/code/lstm/src/confidence_estimation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e377a9696d9833a7"}},{"code_sha256_prefix":"7f20db9fa1afeca3","entry":"posterior_based_conf","repo":"arkilpatel/compositional-generalization-seq2seq","repo_kind":"official","path":"code/lstm/src/confidence_estimation.py","file_url":"https://github.com/arkilpatel/compositional-generalization-seq2seq/blob/HEAD/code/lstm/src/confidence_estimation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7f20db9fa1afeca3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}