{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dual-recurrent-attention-units-for-visual","title":"Dual Recurrent Attention Units for Visual Question Answering","arxiv_id":"1802.00209","date":"2018-02-01","proceeding":null,"authors":["Ahmed Osman","Wojciech Samek"],"abstract":"Visual Question Answering (VQA) requires AI models to comprehend data in two\ndomains, vision and text. Current state-of-the-art models use learned attention\nmechanisms to extract relevant information from the input domains to answer a\ncertain question. Thus, robust attention mechanisms are essential for powerful\nVQA models. In this paper, we propose a recurrent attention mechanism and show\nits benefits compared to the traditional convolutional approach. We perform two\nablation studies to evaluate recurrent attention. First, we introduce a\nbaseline VQA model with visual attention and test the performance difference\nbetween convolutional and recurrent attention on the VQA 2.0 dataset. Secondly,\nwe design an architecture for VQA which utilizes dual (textual and visual)\nRecurrent Attention Units (RAUs). Using this model, we show the effect of all\npossible combinations of recurrent and convolutional dual attention. Our single\nmodel outperforms the first place winner on the VQA 2016 challenge and to the\nbest of our knowledge, it is the second best performing single model on the VQA\n1.0 dataset. Furthermore, our model noticeably improves upon the winner of the\nVQA 2017 challenge. Moreover, we experiment replacing attention mechanisms in\nstate-of-the-art models with our RAUs and show increased performance.","url_abs":"http://arxiv.org/abs/1802.00209v3","url_pdf":"http://arxiv.org/pdf/1802.00209v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dual-recurrent-attention-units-for-visual","repo_url":"https://github.com/ahmedmagdiosman/compress-vqa","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}