{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/end-to-end-dense-video-captioning-as-sequence-1","title":"End-to-end Dense Video Captioning as Sequence Generation","arxiv_id":"2204.08121","date":"2022-04-18","proceeding":"COLING 2022 10","authors":["Wanrong Zhu","Bo Pang","Ashish V. Thapliyal","William Yang Wang","Radu Soricut"],"abstract":"Dense video captioning aims to identify the events of interest in an input video, and generate descriptive captions for each event. Previous approaches usually follow a two-stage generative process, which first proposes a segment for each event, then renders a caption for each identified segment. Recent advances in large-scale sequence generation pretraining have seen great success in unifying task formulation for a great variety of tasks, but so far, more complex tasks such as dense video captioning are not able to fully utilize this powerful paradigm. In this work, we show how to model the two subtasks of dense video captioning jointly as one sequence generation task, and simultaneously predict the events and the corresponding descriptions. Experiments on YouCook2 and ViTT show encouraging results and indicate the feasibility of training complex tasks such as end-to-end dense video captioning integrated into large-scale pretrained models.","url_abs":"https://arxiv.org/abs/2204.08121v2","url_pdf":"https://arxiv.org/pdf/2204.08121v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"dense-video-captioning","task_name":"Dense Video Captioning"},{"task_slug":"descriptive","task_name":"Descriptive"},{"task_slug":"video-captioning","task_name":"Video Captioning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/dense-video-captioning-on-vitt","task":"Dense Video Captioning","dataset":"ViTT","model":"E2ESG","rank_in_archive_order":4,"of":4,"metrics":{"CIDEr":"25.0","METEOR":"8.1"},"uses_additional_data":true}],"syntology":{"syntology_url":"https://syntology.ai/paper/2204.08121","atlas_url":"https://app.syntology.ai/?focus=2204.08121","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}