{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/how2-a-large-scale-dataset-for-multimodal","title":"How2: A Large-scale Dataset for Multimodal Language Understanding","arxiv_id":"1811.00347","date":"2018-11-01","proceeding":null,"authors":["Ramon Sanabria","Ozan Caglayan","Shruti Palaskar","Desmond Elliott","Loïc Barrault","Lucia Specia","Florian Metze"],"abstract":"In this paper, we introduce How2, a multimodal collection of instructional\nvideos with English subtitles and crowdsourced Portuguese translations. We also\npresent integrated sequence-to-sequence baselines for machine translation,\nautomatic speech recognition, spoken language translation, and multimodal\nsummarization. By making available data and code for several multimodal natural\nlanguage tasks, we hope to stimulate more research on these and similar\nchallenges, to obtain a deeper understanding of multimodality in language\nprocessing.","url_abs":"http://arxiv.org/abs/1811.00347v2","url_pdf":"http://arxiv.org/pdf/1811.00347v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"how2-a-large-scale-dataset-for-multimodal","repo_url":"https://github.com/srvk/how2-dataset","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"how2-a-large-scale-dataset-for-multimodal","repo_url":"https://github.com/jasonppy/promptingwhisper","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"automatic-speech-recognition-2","task_name":"Automatic Speech Recognition"},{"task_slug":"automatic-speech-recognition","task_name":"Automatic Speech Recognition (ASR)"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"speech-recognition","task_name":"Speech Recognition"},{"task_slug":"translation","task_name":"Translation"},{"task_slug":"speech-recognition-1","task_name":"speech-recognition"}],"methods":[],"datasets_introduced":[{"slug":"how2","name":"How2","full_name":""}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1811.00347","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}