{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bidirectional-attentive-fusion-with-context","title":"Bidirectional Attentive Fusion with Context Gating for Dense Video Captioning","arxiv_id":"1804.00100","date":"2018-03-31","proceeding":"CVPR 2018 6","authors":["Jingwen Wang","Wenhao Jiang","Lin Ma","Wei Liu","Yong Xu"],"abstract":"Dense video captioning is a newly emerging task that aims at both localizing\nand describing all events in a video. We identify and tackle two challenges on\nthis task, namely, (1) how to utilize both past and future contexts for\naccurate event proposal predictions, and (2) how to construct informative input\nto the decoder for generating natural event descriptions. First, previous works\npredominantly generate temporal event proposals in the forward direction, which\nneglects future video context. We propose a bidirectional proposal method that\neffectively exploits both past and future contexts to make proposal\npredictions. Second, different events ending at (nearly) the same time are\nindistinguishable in the previous works, resulting in the same captions. We\nsolve this problem by representing each event with an attentive fusion of\nhidden states from the proposal module and video contents (e.g., C3D features).\nWe further propose a novel context gating mechanism to balance the\ncontributions from the current event and its surrounding contexts dynamically.\nWe empirically show that our attentively fused event representation is superior\nto the proposal hidden states or video contents alone. By coupling proposal and\ncaptioning modules into one unified framework, our model outperforms the\nstate-of-the-arts on the ActivityNet Captions dataset with a relative gain of\nover 100% (Meteor score increases from 4.82 to 9.65).","url_abs":"http://arxiv.org/abs/1804.00100v2","url_pdf":"http://arxiv.org/pdf/1804.00100v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bidirectional-attentive-fusion-with-context","repo_url":"https://github.com/JaywongWang/DenseVideoCaptioning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"dense-video-captioning","task_name":"Dense Video Captioning"},{"task_slug":"video-captioning","task_name":"Video Captioning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1804.00100","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}