{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-visual-question-answering-by","title":"Learning Visual Question Answering by Bootstrapping Hard Attention","arxiv_id":"1808.00300","date":"2018-08-01","proceeding":"ECCV 2018 9","authors":["Mateusz Malinowski","Carl Doersch","Adam Santoro","Peter Battaglia"],"abstract":"Attention mechanisms in biological perception are thought to select subsets\nof perceptual information for more sophisticated processing which would be\nprohibitive to perform on all sensory inputs. In computer vision, however,\nthere has been relatively little exploration of hard attention, where some\ninformation is selectively ignored, in spite of the success of soft attention,\nwhere information is re-weighted and aggregated, but never filtered out. Here,\nwe introduce a new approach for hard attention and find it achieves very\ncompetitive performance on a recently-released visual question answering\ndatasets, equalling and in some cases surpassing similar soft attention\narchitectures while entirely ignoring some features. Even though the hard\nattention mechanism is thought to be non-differentiable, we found that the\nfeature magnitudes correlate with semantic relevance, and provide a useful\nsignal for our mechanism's attentional selection criterion. Because hard\nattention selects important features of the input information, it can also be\nmore efficient than analogous soft attention mechanisms. This is especially\nimportant for recent approaches that use non-local pairwise operations, whereby\ncomputational and memory costs are quadratic in the size of the set of\nfeatures.","url_abs":"http://arxiv.org/abs/1808.00300v1","url_pdf":"http://arxiv.org/pdf/1808.00300v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-visual-question-answering-by","repo_url":"https://github.com/lienchibao1998/new","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":{"status":"empty_repo"}}],"tasks":[{"task_slug":"hard-attention","task_name":"Hard Attention"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/visual-question-answering-on-clevr","task":"Visual Question Answering (VQA)","dataset":"CLEVR","model":"CNN + LSTM + RN + HAN","rank_in_archive_order":8,"of":15,"metrics":{"Accuracy":"98.8"},"uses_additional_data":false},{"leaderboard":"/sota/visual-question-answering-on-vqa-cp","task":"Visual Question Answering (VQA)","dataset":"VQA-CP","model":"HAN","rank_in_archive_order":10,"of":10,"metrics":{"Score":"28.65"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1808.00300","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}