{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/contextualized-scene-imagination-for-1","title":"Contextualized Scene Imagination for Generative Commonsense Reasoning","arxiv_id":"2112.06318","date":"2021-12-12","proceeding":"ICLR 2022 4","authors":["Peifeng Wang","Jonathan Zamora","Junfeng Liu","Filip Ilievski","Muhao Chen","Xiang Ren"],"abstract":"Humans use natural language to compose common concepts from their environment into plausible, day-to-day scene descriptions. However, such generative commonsense reasoning (GCSR) skills are lacking in state-of-the-art text generation methods. Descriptive sentences about arbitrary concepts generated by neural text generation models (e.g., pre-trained text-to-text Transformers) are often grammatically fluent but may not correspond to human common sense, largely due to their lack of mechanisms to capture concept relations, to identify implicit concepts, and to perform generalizable reasoning about unseen concept compositions. In this paper, we propose an Imagine-and-Verbalize (I&V) method, which learns to imagine a relational scene knowledge graph (SKG) with relations between the input concepts, and leverage the SKG as a constraint when generating a plausible scene description. We collect and harmonize a set of knowledge resources from different domains and modalities, providing a rich auxiliary supervision signal for I&V. The experiments demonstrate the effectiveness of I&V in improving language models on both concept-to-sentence and concept-to-story generation tasks, while enabling the model to learn well from fewer task examples and generate SKGs that make common sense to human annotators.","url_abs":"https://arxiv.org/abs/2112.06318v3","url_pdf":"https://arxiv.org/pdf/2112.06318v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"contextualized-scene-imagination-for-1","repo_url":"https://github.com/wangpf3/imagine-and-verbalize","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"common-sense-reasoning","task_name":"Common Sense Reasoning"},{"task_slug":"descriptive","task_name":"Descriptive"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"story-generation","task_name":"Story Generation"},{"task_slug":"text-generation","task_name":"Text Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2112.06318","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2112.06318"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/wangpf3/imagine-and-verbalize","reach":null}],"summary":{"ran":1,"unverified":3},"by_repo_kind":{"official":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"e771f361f3766cc1","entry":"LabelSmoothingLoss","repo":"wangpf3/imagine-and-verbalize","repo_kind":"official","path":"imagination_learning/lib/model.py","file_url":"https://github.com/wangpf3/imagine-and-verbalize/blob/HEAD/imagination_learning/lib/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e771f361f3766cc1"}},{"code_sha256_prefix":"49427e81994d80e0","entry":"Build_Model","repo":"wangpf3/imagine-and-verbalize","repo_kind":"official","path":"imagination_learning/lib/model.py","file_url":"https://github.com/wangpf3/imagine-and-verbalize/blob/HEAD/imagination_learning/lib/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"49427e81994d80e0"}},{"code_sha256_prefix":"66d28517942d476e","entry":"Data_Helper","repo":"wangpf3/imagine-and-verbalize","repo_kind":"official","path":"imagination_learning/lib/model.py","file_url":"https://github.com/wangpf3/imagine-and-verbalize/blob/HEAD/imagination_learning/lib/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"66d28517942d476e"}},{"code_sha256_prefix":"c958be3b74817df7","entry":"get_logger","repo":"wangpf3/imagine-and-verbalize","repo_kind":"official","path":"imagination_learning/lib/model.py","file_url":"https://github.com/wangpf3/imagine-and-verbalize/blob/HEAD/imagination_learning/lib/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c958be3b74817df7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}