{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/end-to-end-dense-video-captioning-with","title":"End-to-End Dense Video Captioning with Parallel Decoding","arxiv_id":"2108.07781","date":"2021-08-17","proceeding":"ICCV 2021 10","authors":["Teng Wang","Ruimao Zhang","Zhichao Lu","Feng Zheng","Ran Cheng","Ping Luo"],"abstract":"Dense video captioning aims to generate multiple associated captions with their temporal locations from the video. Previous methods follow a sophisticated \"localize-then-describe\" scheme, which heavily relies on numerous hand-crafted components. In this paper, we proposed a simple yet effective framework for end-to-end dense video captioning with parallel decoding (PDVC), by formulating the dense caption generation as a set prediction task. In practice, through stacking a newly proposed event counter on the top of a transformer decoder, the PDVC precisely segments the video into a number of event pieces under the holistic understanding of the video content, which effectively increases the coherence and readability of predicted captions. Compared with prior arts, the PDVC has several appealing advantages: (1) Without relying on heuristic non-maximum suppression or a recurrent event sequence selection network to remove redundancy, PDVC directly produces an event set with an appropriate size; (2) In contrast to adopting the two-stage scheme, we feed the enhanced representations of event queries into the localization head and caption head in parallel, making these two sub-tasks deeply interrelated and mutually promoted through the optimization; (3) Without bells and whistles, extensive experiments on ActivityNet Captions and YouCook2 show that PDVC is capable of producing high-quality captioning results, surpassing the state-of-the-art two-stage methods when its localization accuracy is on par with them. Code is available at https://github.com/ttengwang/PDVC.","url_abs":"https://arxiv.org/abs/2108.07781v2","url_pdf":"https://arxiv.org/pdf/2108.07781v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"end-to-end-dense-video-captioning-with","repo_url":"https://github.com/ttengwang/pdvc","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"end-to-end-dense-video-captioning-with","repo_url":"https://github.com/aim3-ruc/youmakeup_challenge2022","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"caption-generation","task_name":"Caption Generation"},{"task_slug":"dense-video-captioning","task_name":"Dense Video Captioning"},{"task_slug":"video-captioning","task_name":"Video Captioning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/dense-video-captioning-on-activitynet","task":"Dense Video Captioning","dataset":"ActivityNet Captions","model":"PDVC (TSP features, no SCST)","rank_in_archive_order":6,"of":12,"metrics":{"BLEU-4":"2.17","CIDEr":"31.14","METEOR":"9.03","SODA":"6.05"},"uses_additional_data":false},{"leaderboard":"/sota/dense-video-captioning-on-youcook2","task":"Dense Video Captioning","dataset":"YouCook2","model":"PDVC (TSN features, no SCST)","rank_in_archive_order":6,"of":7,"metrics":{"BLEU4":"0.8","CIDEr":"22.71","METEOR":"4.74","SODA":"4.42"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2108.07781","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2108.07781"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ttengwang/pdvc","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/aim3-ruc/youmakeup_challenge2022","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/ttengwang/PDVC","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"official":{"samples":7,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"88861105cb84eccd","entry":"decide_two_stage","repo":"ttengwang/PDVC","repo_kind":"official","path":"pdvc/pdvc.py","file_url":"https://github.com/ttengwang/PDVC/blob/HEAD/pdvc/pdvc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"88861105cb84eccd"}},{"code_sha256_prefix":"65a13ac177e73bcd","entry":"regression_loss","repo":"ttengwang/pdvc","repo_kind":"official","path":"pdvc/criterion.py","file_url":"https://github.com/ttengwang/pdvc/blob/HEAD/pdvc/criterion.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"65a13ac177e73bcd"}},{"code_sha256_prefix":"5c0711aada67957e","entry":"sigmoid_focal_loss","repo":"ttengwang/pdvc","repo_kind":"official","path":"pdvc/criterion.py","file_url":"https://github.com/ttengwang/pdvc/blob/HEAD/pdvc/criterion.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5c0711aada67957e"}},{"code_sha256_prefix":"9f7f34125123e431","entry":"PDVC","repo":"ttengwang/PDVC","repo_kind":"official","path":"pdvc/pdvc.py","file_url":"https://github.com/ttengwang/PDVC/blob/HEAD/pdvc/pdvc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9f7f34125123e431"}},{"code_sha256_prefix":"53e64888999b6cb7","entry":"calculate_avg_proposal_num","repo":"ttengwang/pdvc","repo_kind":"official","path":"eval_utils.py","file_url":"https://github.com/ttengwang/pdvc/blob/HEAD/eval_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"53e64888999b6cb7"}},{"code_sha256_prefix":"594f41589a9b1b8a","entry":"cross_entropy_with_gaussian_mask","repo":"ttengwang/pdvc","repo_kind":"official","path":"pdvc/criterion.py","file_url":"https://github.com/ttengwang/pdvc/blob/HEAD/pdvc/criterion.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"594f41589a9b1b8a"}},{"code_sha256_prefix":"bf97e726097b2a5f","entry":"reranking","repo":"ttengwang/pdvc","repo_kind":"official","path":"eval_utils.py","file_url":"https://github.com/ttengwang/pdvc/blob/HEAD/eval_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bf97e726097b2a5f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}