{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dont-just-assume-look-and-answer-overcoming","title":"Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering","arxiv_id":"1712.00377","date":"2017-12-01","proceeding":"CVPR 2018 6","authors":["Aishwarya Agrawal","Dhruv Batra","Devi Parikh","Aniruddha Kembhavi"],"abstract":"A number of studies have found that today's Visual Question Answering (VQA)\nmodels are heavily driven by superficial correlations in the training data and\nlack sufficient image grounding. To encourage development of models geared\ntowards the latter, we propose a new setting for VQA where for every question\ntype, train and test sets have different prior distributions of answers.\nSpecifically, we present new splits of the VQA v1 and VQA v2 datasets, which we\ncall Visual Question Answering under Changing Priors (VQA-CP v1 and VQA-CP v2\nrespectively). First, we evaluate several existing VQA models under this new\nsetting and show that their performance degrades significantly compared to the\noriginal VQA setting. Second, we propose a novel Grounded Visual Question\nAnswering model (GVQA) that contains inductive biases and restrictions in the\narchitecture specifically designed to prevent the model from 'cheating' by\nprimarily relying on priors in the training data. Specifically, GVQA explicitly\ndisentangles the recognition of visual concepts present in the image from the\nidentification of plausible answer space for a given question, enabling the\nmodel to more robustly generalize across different distributions of answers.\nGVQA is built off an existing VQA model -- Stacked Attention Networks (SAN).\nOur experiments demonstrate that GVQA significantly outperforms SAN on both\nVQA-CP v1 and VQA-CP v2 datasets. Interestingly, it also outperforms more\npowerful VQA models such as Multimodal Compact Bilinear Pooling (MCB) in\nseveral cases. GVQA offers strengths complementary to SAN when trained and\nevaluated on the original VQA v1 and VQA v2 datasets. Finally, GVQA is more\ntransparent and interpretable than existing VQA models.","url_abs":"http://arxiv.org/abs/1712.00377v2","url_pdf":"http://arxiv.org/pdf/1712.00377v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dont-just-assume-look-and-answer-overcoming","repo_url":"https://github.com/AishwaryaAgrawal/GVQA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[],"datasets_introduced":[{"slug":"vqa-cp","name":"VQA-CP","full_name":null}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1712.00377","atlas_url":"https://app.syntology.ai/?focus=1712.00377","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}