{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-convolutional-text-representations","title":"Learning Convolutional Text Representations for Visual Question Answering","arxiv_id":"1705.06824","date":"2017-05-18","proceeding":null,"authors":["Zhengyang Wang","Shuiwang Ji"],"abstract":"Visual question answering is a recently proposed artificial intelligence task\nthat requires a deep understanding of both images and texts. In deep learning,\nimages are typically modeled through convolutional neural networks, and texts\nare typically modeled through recurrent neural networks. While the requirement\nfor modeling images is similar to traditional computer vision tasks, such as\nobject recognition and image classification, visual question answering raises a\ndifferent need for textual representation as compared to other natural language\nprocessing tasks. In this work, we perform a detailed analysis on natural\nlanguage questions in visual question answering. Based on the analysis, we\npropose to rely on convolutional neural networks for learning textual\nrepresentations. By exploring the various properties of convolutional neural\nnetworks specialized for text data, such as width and depth, we present our\n\"CNN Inception + Gate\" model. We show that our model improves question\nrepresentations and thus the overall accuracy of visual question answering\nmodels. We also show that the text representation requirement in visual\nquestion answering is more complicated and comprehensive than that in\nconventional natural language processing tasks, making it a better task to\nevaluate textual representation methods. Shallow models like fastText, which\ncan obtain comparable results with deep learning models in tasks like text\nclassification, are not suitable in visual question answering.","url_abs":"http://arxiv.org/abs/1705.06824v2","url_pdf":"http://arxiv.org/pdf/1705.06824v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-convolutional-text-representations","repo_url":"https://github.com/divelab/vqa-text","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"caffe2","reach":{"status":"ok"}}],"tasks":[{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"},{"task_slug":"image-classification","task_name":"image-classification"},{"task_slug":"text-classification-1","task_name":"text-classification"}],"methods":[{"method_slug":"fasttext","method_name":"fastText"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}