{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/compositional-generalization-in-a-deep","title":"Compositional generalization in a deep seq2seq model by separating syntax and semantics","arxiv_id":"1904.09708","date":"2019-04-22","proceeding":null,"authors":["Jake Russin","Jason Jo","Randall C. O'Reilly","Yoshua Bengio"],"abstract":"Standard methods in deep learning for natural language processing fail to capture the compositional structure of human language that allows for systematic generalization outside of the training distribution. However, human learners readily generalize in this way, e.g. by applying known grammatical rules to novel words. Inspired by work in neuroscience suggesting separate brain systems for syntactic and semantic processing, we implement a modification to standard approaches in neural machine translation, imposing an analogous separation. The novel model, which we call Syntactic Attention, substantially outperforms standard methods in deep learning on the SCAN dataset, a compositional generalization task, without any hand-engineered features or additional supervision. Our work suggests that separating syntactic from semantic learning may be a useful heuristic for capturing compositional structure.","url_abs":"https://arxiv.org/abs/1904.09708v3","url_pdf":"https://arxiv.org/pdf/1904.09708v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"compositional-generalization-in-a-deep","repo_url":"https://github.com/jlrussin/syntactic_attention","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"systematic-generalization","task_name":"Systematic Generalization"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1904.09708","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1904.09708"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jlrussin/syntactic_attention","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":4},"by_repo_kind":{"listed":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a86f589eb0c79620","entry":"SCAN_collate","repo":"jlrussin/syntactic_attention","repo_kind":"listed","path":"data.py","file_url":"https://github.com/jlrussin/syntactic_attention/blob/HEAD/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a86f589eb0c79620"}},{"code_sha256_prefix":"e4e112cc7199ed10","entry":"evaluate_bleu","repo":"jlrussin/syntactic_attention","repo_kind":"listed","path":"evaluate_bleu.py","file_url":"https://github.com/jlrussin/syntactic_attention/blob/HEAD/evaluate_bleu.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e4e112cc7199ed10"}},{"code_sha256_prefix":"e715528c3e691cc9","entry":"get_reference_dict","repo":"jlrussin/syntactic_attention","repo_kind":"listed","path":"evaluate_bleu.py","file_url":"https://github.com/jlrussin/syntactic_attention/blob/HEAD/evaluate_bleu.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e715528c3e691cc9"}},{"code_sha256_prefix":"8d0627fbb5ef1bca","entry":"str2bool","repo":"jlrussin/syntactic_attention","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/jlrussin/syntactic_attention/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8d0627fbb5ef1bca"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}