{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/structured-attentions-for-visual-question","title":"Structured Attentions for Visual Question Answering","arxiv_id":"1708.02071","date":"2017-08-07","proceeding":"ICCV 2017 10","authors":["Chen Zhu","Yanpeng Zhao","Shuaiyi Huang","Kewei Tu","Yi Ma"],"abstract":"Visual attention, which assigns weights to image regions according to their\nrelevance to a question, is considered as an indispensable part by most Visual\nQuestion Answering models. Although the questions may involve complex relations\namong multiple regions, few attention models can effectively encode such\ncross-region relations. In this paper, we demonstrate the importance of\nencoding such relations by showing the limited effective receptive field of\nResNet on two datasets, and propose to model the visual attention as a\nmultivariate distribution over a grid-structured Conditional Random Field on\nimage regions. We demonstrate how to convert the iterative inference\nalgorithms, Mean Field and Loopy Belief Propagation, as recurrent layers of an\nend-to-end neural network. We empirically evaluated our model on 3 datasets, in\nwhich it surpasses the best baseline model of the newly released CLEVR dataset\nby 9.5%, and the best published model on the VQA dataset by 1.25%. Source code\nis available at https: //github.com/zhuchen03/vqa-sva.","url_abs":"http://arxiv.org/abs/1708.02071v1","url_pdf":"http://arxiv.org/pdf/1708.02071v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"structured-attentions-for-visual-question","repo_url":"https://github.com/zhuchen03/vqa-sva","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}