{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/clevr-dialog-a-diagnostic-dataset-for-multi","title":"CLEVR-Dialog: A Diagnostic Dataset for Multi-Round Reasoning in Visual Dialog","arxiv_id":"1903.03166","date":"2019-03-07","proceeding":"NAACL 2019 6","authors":["Satwik Kottur","José M. F. Moura","Devi Parikh","Dhruv Batra","Marcus Rohrbach"],"abstract":"Visual Dialog is a multimodal task of answering a sequence of questions grounded in an image, using the conversation history as context. It entails challenges in vision, language, reasoning, and grounding. However, studying these subtasks in isolation on large, real datasets is infeasible as it requires prohibitively-expensive complete annotation of the 'state' of all images and dialogs. We develop CLEVR-Dialog, a large diagnostic dataset for studying multi-round reasoning in visual dialog. Specifically, we construct a dialog grammar that is grounded in the scene graphs of the images from the CLEVR dataset. This combination results in a dataset where all aspects of the visual dialog are fully annotated. In total, CLEVR-Dialog contains 5 instances of 10-round dialogs for about 85k CLEVR images, totaling to 4.25M question-answer pairs. We use CLEVR-Dialog to benchmark performance of standard visual dialog models; in particular, on visual coreference resolution (as a function of the coreference distance). This is the first analysis of its kind for visual dialog models that was not possible without this dataset. We hope the findings from CLEVR-Dialog will help inform the development of future models for visual dialog. Our dataset and code are publicly available.","url_abs":"https://arxiv.org/abs/1903.03166v2","url_pdf":"https://arxiv.org/pdf/1903.03166v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"clevr-dialog-a-diagnostic-dataset-for-multi","repo_url":"https://github.com/satwikkottur/clevr-dialog","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"coreference-resolution","task_name":"Coreference Resolution"},{"task_slug":"diagnostic","task_name":"Diagnostic"},{"task_slug":"visual-dialogue","task_name":"Visual Dialog"},{"task_slug":"coreference-resolution-1","task_name":"coreference-resolution"}],"methods":[],"datasets_introduced":[{"slug":"clevr-dialog","name":"CLEVR-Dialog","full_name":"CLEVR-Dialog"}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1903.03166","atlas_url":"https://app.syntology.ai/?focus=1903.03166","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1903.03166"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/satwikkottur/clevr-dialog","reach":null}],"summary":{"ran_honours":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ea169fbffa0825f5","entry":"get_tag_group","repo":"satwikkottur/clevr-dialog","repo_kind":"official","path":"generate_dataset.py","file_url":"https://github.com/satwikkottur/clevr-dialog/blob/HEAD/generate_dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"ea169fbffa0825f5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}