{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/reascan-compositional-reasoning-in-language","title":"ReaSCAN: Compositional Reasoning in Language Grounding","arxiv_id":"2109.08994","date":"2021-09-18","proceeding":null,"authors":["Zhengxuan Wu","Elisa Kreiss","Desmond C. Ong","Christopher Potts"],"abstract":"The ability to compositionally map language to referents, relations, and actions is an essential component of language understanding. The recent gSCAN dataset (Ruis et al. 2020, NeurIPS) is an inspiring attempt to assess the capacity of models to learn this kind of grounding in scenarios involving navigational instructions. However, we show that gSCAN's highly constrained design means that it does not require compositional interpretation and that many details of its instructions and scenarios are not required for task success. To address these limitations, we propose ReaSCAN, a benchmark dataset that builds off gSCAN but requires compositional language interpretation and reasoning about entities and relations. We assess two models on ReaSCAN: a multi-modal baseline and a state-of-the-art graph convolutional neural model. These experiments show that ReaSCAN is substantially harder than gSCAN for both neural architectures. This suggests that ReaSCAN can serve as a valuable benchmark for advancing our understanding of models' compositional generalization and reasoning capabilities.","url_abs":"https://arxiv.org/abs/2109.08994v1","url_pdf":"https://arxiv.org/pdf/2109.08994v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"reascan-compositional-reasoning-in-language","repo_url":"https://github.com/frankaging/Reason-SCAN","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"reascan-compositional-reasoning-in-language","repo_url":"https://github.com/ankursikarwar/grounded-compositional-generalization","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"reascan-compositional-reasoning-in-language","repo_url":"https://github.com/frankaging/Causal-Distill","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[{"slug":"reascan","name":"ReaSCAN","full_name":"ReaSCAN: Compositional Reasoning in Language Grounding"}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2109.08994","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2109.08994"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ankursikarwar/grounded-compositional-generalization","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/frankaging/Reason-SCAN","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/frankaging/Causal-Distill","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"7ac46edb64b659d9","entry":"get_attribute_statistics","repo":"frankaging/Reason-SCAN","repo_kind":"official","path":"code/dataset/generate_ReaSCAN.py","file_url":"https://github.com/frankaging/Reason-SCAN/blob/HEAD/code/dataset/generate_ReaSCAN.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"7ac46edb64b659d9"}},{"code_sha256_prefix":"9d1cdb2384d16002","entry":"get_keyword_statistics","repo":"frankaging/Reason-SCAN","repo_kind":"official","path":"code/dataset/generate_ReaSCAN.py","file_url":"https://github.com/frankaging/Reason-SCAN/blob/HEAD/code/dataset/generate_ReaSCAN.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"9d1cdb2384d16002"}},{"code_sha256_prefix":"485ce69024a9e0f8","entry":"get_relation_statistics","repo":"frankaging/Reason-SCAN","repo_kind":"official","path":"code/dataset/generate_ReaSCAN.py","file_url":"https://github.com/frankaging/Reason-SCAN/blob/HEAD/code/dataset/generate_ReaSCAN.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"485ce69024a9e0f8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}