{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/an-analysis-of-visual-question-answering","title":"An Analysis of Visual Question Answering Algorithms","arxiv_id":"1703.09684","date":"2017-03-28","proceeding":"ICCV 2017 10","authors":["Kushal Kafle","Christopher Kanan"],"abstract":"In visual question answering (VQA), an algorithm must answer text-based\nquestions about images. While multiple datasets for VQA have been created since\nlate 2014, they all have flaws in both their content and the way algorithms are\nevaluated on them. As a result, evaluation scores are inflated and\npredominantly determined by answering easier questions, making it difficult to\ncompare different methods. In this paper, we analyze existing VQA algorithms\nusing a new dataset. It contains over 1.6 million questions organized into 12\ndifferent categories. We also introduce questions that are meaningless for a\ngiven image to force a VQA system to reason about image content. We propose new\nevaluation schemes that compensate for over-represented question-types and make\nit easier to study the strengths and weaknesses of algorithms. We analyze the\nperformance of both baseline and state-of-the-art VQA models, including\nmulti-modal compact bilinear pooling (MCB), neural module networks, and\nrecurrent answering units. Our experiments establish how attention helps\ncertain categories more than others, determine which models work better than\nothers, and explain how simple models (e.g. MLP) can surpass more complex\nmodels (MCB) by simply learning to answer large, easy question categories.","url_abs":"http://arxiv.org/abs/1703.09684v2","url_pdf":"http://arxiv.org/pdf/1703.09684v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[],"datasets_introduced":[{"slug":"tdiuc","name":"TDIUC","full_name":"Task Directed Image Understanding Challenge"}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1703.09684","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}