{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/lexicon-learning-for-few-shot-neural-sequence","title":"Lexicon Learning for Few-Shot Neural Sequence Modeling","arxiv_id":"2106.03993","date":"2021-06-07","proceeding":null,"authors":["Ekin Akyürek","Jacob Andreas"],"abstract":"Sequence-to-sequence transduction is the core problem in language processing applications as diverse as semantic parsing, machine translation, and instruction following. The neural network models that provide the dominant solution to these problems are brittle, especially in low-resource settings: they fail to generalize correctly or systematically from small datasets. Past work has shown that many failures of systematic generalization arise from neural models' inability to disentangle lexical phenomena from syntactic ones. To address this, we augment neural decoders with a lexical translation mechanism that generalizes existing copy mechanisms to incorporate learned, decontextualized, token-level translation rules. We describe how to initialize this mechanism using a variety of lexicon learning algorithms, and show that it improves systematic generalization on a diverse set of sequence modeling tasks drawn from cognitive science, formal semantics, and machine translation.","url_abs":"https://arxiv.org/abs/2106.03993v1","url_pdf":"https://arxiv.org/pdf/2106.03993v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"lexicon-learning-for-few-shot-neural-sequence","repo_url":"https://github.com/ekinakyurek/lexical","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"instruction-following","task_name":"Instruction Following"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"semantic-parsing","task_name":"Semantic Parsing"},{"task_slug":"systematic-generalization","task_name":"Systematic Generalization"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2106.03993","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.03993"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ekinakyurek/lexical","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":4,"unverified":4},"by_repo_kind":{"official":{"samples":8,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"839530fa476fdb3b","entry":"encode","repo":"ekinakyurek/lexical","repo_kind":"official","path":"data.py","file_url":"https://github.com/ekinakyurek/lexical/blob/HEAD/data.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"839530fa476fdb3b"}},{"code_sha256_prefix":"82101630969e5b8e","entry":"encode_io","repo":"ekinakyurek/lexical","repo_kind":"official","path":"data.py","file_url":"https://github.com/ekinakyurek/lexical/blob/HEAD/data.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"82101630969e5b8e"}},{"code_sha256_prefix":"7cb6c55af2773820","entry":"fn","repo":"ekinakyurek/lexical","repo_kind":"official","path":"hlog.py","file_url":"https://github.com/ekinakyurek/lexical/blob/HEAD/hlog.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7cb6c55af2773820"}},{"code_sha256_prefix":"f672aab27d220c13","entry":"group","repo":"ekinakyurek/lexical","repo_kind":"official","path":"hlog.py","file_url":"https://github.com/ekinakyurek/lexical/blob/HEAD/hlog.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f672aab27d220c13"}},{"code_sha256_prefix":"7841344d2c132f93","entry":"batch_seqs","repo":"ekinakyurek/lexical","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/ekinakyurek/lexical/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7841344d2c132f93"}},{"code_sha256_prefix":"4e04ec15da964ec6","entry":"eval_format","repo":"ekinakyurek/lexical","repo_kind":"official","path":"data.py","file_url":"https://github.com/ekinakyurek/lexical/blob/HEAD/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4e04ec15da964ec6"}},{"code_sha256_prefix":"0ff22e14159fcfd0","entry":"trim","repo":"ekinakyurek/lexical","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/ekinakyurek/lexical/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0ff22e14159fcfd0"}},{"code_sha256_prefix":"eedb78f0f2e7f4e9","entry":"weight_top_p","repo":"ekinakyurek/lexical","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/ekinakyurek/lexical/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eedb78f0f2e7f4e9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}