{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/human-attention-in-visual-question-answering","title":"Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?","arxiv_id":"1606.05589","date":"2016-06-17","proceeding":null,"authors":["Abhishek Das","Harsh Agrawal","C. Lawrence Zitnick","Devi Parikh","Dhruv Batra"],"abstract":"We conduct large-scale studies on `human attention' in Visual Question\nAnswering (VQA) to understand where humans choose to look to answer questions\nabout images. We design and test multiple game-inspired novel\nattention-annotation interfaces that require the subject to sharpen regions of\na blurred image to answer a question. Thus, we introduce the VQA-HAT (Human\nATtention) dataset. We evaluate attention maps generated by state-of-the-art\nVQA models against human attention both qualitatively (via visualizations) and\nquantitatively (via rank-order correlation). Overall, our experiments show that\ncurrent attention models in VQA do not seem to be looking at the same regions\nas humans.","url_abs":"http://arxiv.org/abs/1606.05589v1","url_pdf":"http://arxiv.org/pdf/1606.05589v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[],"datasets_introduced":[{"slug":"vqa-hat","name":"VQA-HAT","full_name":"VQA Human Attention"}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1606.05589","atlas_url":"https://app.syntology.ai/?focus=1606.05589","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}