{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/are-you-talking-to-me-reasoned-visual-dialog","title":"Are You Talking to Me? Reasoned Visual Dialog Generation through Adversarial Learning","arxiv_id":"1711.07613","date":"2017-11-21","proceeding":"CVPR 2018 6","authors":["Qi Wu","Peng Wang","Chunhua Shen","Ian Reid","Anton Van Den Hengel"],"abstract":"The Visual Dialogue task requires an agent to engage in a conversation about\nan image with a human. It represents an extension of the Visual Question\nAnswering task in that the agent needs to answer a question about an image, but\nit needs to do so in light of the previous dialogue that has taken place. The\nkey challenge in Visual Dialogue is thus maintaining a consistent, and natural\ndialogue while continuing to answer questions correctly. We present a novel\napproach that combines Reinforcement Learning and Generative Adversarial\nNetworks (GANs) to generate more human-like responses to questions. The GAN\nhelps overcome the relative paucity of training data, and the tendency of the\ntypical MLE-based approach to generate overly terse answers. Critically, the\nGAN is tightly integrated into the attention mechanism that generates\nhuman-interpretable reasons for each answer. This means that the discriminative\nmodel of the GAN has the task of assessing whether a candidate answer is\ngenerated by a human or not, given the provided reason. This is significant\nbecause it drives the generative model to produce high quality answers that are\nwell supported by the associated reasoning. The method also generates the\nstate-of-the-art results on the primary benchmark.","url_abs":"http://arxiv.org/abs/1711.07613v1","url_pdf":"http://arxiv.org/pdf/1711.07613v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"visual-dialogue","task_name":"Visual Dialog"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/visual-dialog-on-visdial-v09-val","task":"Visual Dialog","dataset":"VisDial v0.9 val","model":"CoAtt","rank_in_archive_order":4,"of":18,"metrics":{"MRR":"63.98","Mean Rank":"4.47","R@1":"50.29","R@10":"88.81","R@5":"80.71"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1711.07613","atlas_url":"https://app.syntology.ai/?focus=1711.07613","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}