{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/question-guided-hybrid-convolution-for-visual","title":"Question-Guided Hybrid Convolution for Visual Question Answering","arxiv_id":"1808.02632","date":"2018-08-08","proceeding":"ECCV 2018 9","authors":["Peng Gao","Pan Lu","Hongsheng Li","Shuang Li","Yikang Li","Steven Hoi","Xiaogang Wang"],"abstract":"In this paper, we propose a novel Question-Guided Hybrid Convolution (QGHC)\nnetwork for Visual Question Answering (VQA). Most state-of-the-art VQA methods\nfuse the high-level textual and visual features from the neural network and\nabandon the visual spatial information when learning multi-modal features.To\naddress these problems, question-guided kernels generated from the input\nquestion are designed to convolute with visual features for capturing the\ntextual and visual relationship in the early stage. The question-guided\nconvolution can tightly couple the textual and visual information but also\nintroduce more parameters when learning kernels. We apply the group\nconvolution, which consists of question-independent kernels and\nquestion-dependent kernels, to reduce the parameter size and alleviate\nover-fitting. The hybrid convolution can generate discriminative multi-modal\nfeatures with fewer parameters. The proposed approach is also complementary to\nexisting bilinear pooling fusion and attention based VQA methods. By\nintegrating with them, our method could further boost the performance.\nExtensive experiments on public VQA datasets validate the effectiveness of\nQGHC.","url_abs":"http://arxiv.org/abs/1808.02632v1","url_pdf":"http://arxiv.org/pdf/1808.02632v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/visual-question-answering-on-clevr","task":"Visual Question Answering (VQA)","dataset":"CLEVR","model":"QGHC+Att+Concat","rank_in_archive_order":15,"of":15,"metrics":{"Accuracy":"65.90"},"uses_additional_data":false},{"leaderboard":"/sota/visual-question-answering-on-coco-visual-4","task":"Visual Question Answering (VQA)","dataset":"COCO Visual Question Answering (VQA) real images 1.0 open ended","model":"QGHC+Att+Concat","rank_in_archive_order":3,"of":14,"metrics":{"Percentage correct":"65.90"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1808.02632","atlas_url":"https://app.syntology.ai/?focus=1808.02632","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}