{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/generating-natural-questions-about-an-image","title":"Generating Natural Questions About an Image","arxiv_id":"1603.06059","date":"2016-03-19","proceeding":"ACL 2016 8","authors":["Nasrin Mostafazadeh","Ishan Misra","Jacob Devlin","Margaret Mitchell","Xiaodong He","Lucy Vanderwende"],"abstract":"There has been an explosion of work in the vision & language community during\nthe past few years from image captioning to video transcription, and answering\nquestions about images. These tasks have focused on literal descriptions of the\nimage. To move beyond the literal, we choose to explore how questions about an\nimage are often directed at commonsense inference and the abstract events\nevoked by objects in the image. In this paper, we introduce the novel task of\nVisual Question Generation (VQG), where the system is tasked with asking a\nnatural and engaging question when shown an image. We provide three datasets\nwhich cover a variety of images from object-centric to event-centric, with\nconsiderably more abstract training data than provided to state-of-the-art\ncaptioning systems thus far. We train and test several generative and retrieval\nmodels to tackle the task of VQG. Evaluation results show that while such\nmodels ask reasonable questions for a variety of images, there is still a wide\ngap with human performance which motivates further work on connecting images\nwith commonsense knowledge and pragmatics. Our proposed task offers a new\nchallenge to the community which we hope furthers interest in exploring deeper\nconnections between vision & language.","url_abs":"http://arxiv.org/abs/1603.06059v3","url_pdf":"http://arxiv.org/pdf/1603.06059v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"generating-natural-questions-about-an-image","repo_url":"https://github.com/chingyaoc/vqg-tensorflow","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"generating-natural-questions-about-an-image","repo_url":"https://github.com/dreamstorm25/Generating-Natural-Questions-from-an-Image","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"image-captioning","task_name":"Image Captioning"},{"task_slug":"natural-questions","task_name":"Natural Questions"},{"task_slug":"question-generation","task_name":"Question Generation"},{"task_slug":"question-generation","task_name":"Question-Generation"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[],"datasets_introduced":[{"slug":"vqg","name":"VQG","full_name":"Visual Question Generation"}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1603.06059","atlas_url":"https://app.syntology.ai/?focus=1603.06059","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1603.06059"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/chingyaoc/vqg-tensorflow","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dreamstorm25/Generating-Natural-Questions-from-an-Image","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":1},"by_repo_kind":{},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"d39d1cd46009e948","entry":"numericalSort","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"d39d1cd46009e948"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}