{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/game-based-video-context-dialogue","title":"Game-Based Video-Context Dialogue","arxiv_id":"1809.04560","date":"2018-09-12","proceeding":"EMNLP 2018 10","authors":["Ramakanth Pasunuru","Mohit Bansal"],"abstract":"Current dialogue systems focus more on textual and speech context knowledge\nand are usually based on two speakers. Some recent work has investigated static\nimage-based dialogue. However, several real-world human interactions also\ninvolve dynamic visual context (similar to videos) as well as dialogue\nexchanges among multiple speakers. To move closer towards such multimodal\nconversational skills and visually-situated applications, we introduce a new\nvideo-context, many-speaker dialogue dataset based on live-broadcast soccer\ngame videos and chats from Twitch.tv. This challenging testbed allows us to\ndevelop visually-grounded dialogue models that should generate relevant\ntemporal and spatial event language from the live video, while also being\nrelevant to the chat history. For strong baselines, we also present several\ndiscriminative and generative models, e.g., based on tridirectional attention\nflow (TriDAF). We evaluate these models via retrieval ranking-recall, automatic\nphrase-matching metrics, as well as human evaluation studies. We also present\ndataset analyses, model ablations, and visualizations to understand the\ncontribution of different modalities and model components.","url_abs":"http://arxiv.org/abs/1809.04560v2","url_pdf":"http://arxiv.org/pdf/1809.04560v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"game-based-video-context-dialogue","repo_url":"https://github.com/ramakanth-pasunuru/video-dialogue","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[],"datasets_introduced":[{"slug":"twitch-fifa","name":"Twitch-FIFA","full_name":null}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1809.04560","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}