{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unifying-event-detection-and-captioning-as","title":"Unifying Event Detection and Captioning as Sequence Generation via Pre-Training","arxiv_id":"2207.08625","date":"2022-07-18","proceeding":null,"authors":["Qi Zhang","Yuqing Song","Qin Jin"],"abstract":"Dense video captioning aims to generate corresponding text descriptions for a series of events in the untrimmed video, which can be divided into two sub-tasks, event detection and event captioning. Unlike previous works that tackle the two sub-tasks separately, recent works have focused on enhancing the inter-task association between the two sub-tasks. However, designing inter-task interactions for event detection and captioning is not trivial due to the large differences in their task specific solutions. Besides, previous event detection methods normally ignore temporal dependencies between events, leading to event redundancy or inconsistency problems. To tackle above the two defects, in this paper, we define event detection as a sequence generation task and propose a unified pre-training and fine-tuning framework to naturally enhance the inter-task association between event detection and captioning. Since the model predicts each event with previous events as context, the inter-dependency between events is fully exploited and thus our model can detect more diverse and consistent events in the video. Experiments on the ActivityNet dataset show that our model outperforms the state-of-the-art methods, and can be further boosted when pre-trained on extra large-scale video-text data. Code is available at \\url{https://github.com/QiQAng/UEDVC}.","url_abs":"https://arxiv.org/abs/2207.08625v1","url_pdf":"https://arxiv.org/pdf/2207.08625v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unifying-event-detection-and-captioning-as","repo_url":"https://github.com/qiqang/uedvc","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"dense-video-captioning","task_name":"Dense Video Captioning"},{"task_slug":"event-detection","task_name":"Event Detection"},{"task_slug":"video-captioning","task_name":"Video Captioning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2207.08625","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2207.08625"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/QiQAng/UEDVC","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/ranjaykrishna/densevid","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/fujiso/SODA","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran":5,"ran_draft_wrong":2,"unverified":6},"by_repo_kind":{"official":{"samples":13,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":13,"samples":[{"code_sha256_prefix":"b3dcd70203b0bd3e","entry":"Embedder","repo":"QiQAng/UEDVC","repo_kind":"official","path":"modules/transformer.py","file_url":"https://github.com/QiQAng/UEDVC/blob/HEAD/modules/transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b3dcd70203b0bd3e"}},{"code_sha256_prefix":"64ff27ba3fec6464","entry":"FeedForward","repo":"QiQAng/UEDVC","repo_kind":"official","path":"modules/transformer.py","file_url":"https://github.com/QiQAng/UEDVC/blob/HEAD/modules/transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"64ff27ba3fec6464"}},{"code_sha256_prefix":"e2fd5904ba4c6794","entry":"MultiHeadAttention","repo":"QiQAng/UEDVC","repo_kind":"official","path":"modules/transformer.py","file_url":"https://github.com/QiQAng/UEDVC/blob/HEAD/modules/transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e2fd5904ba4c6794"}},{"code_sha256_prefix":"37d512fc00346438","entry":"PositionalEncoder","repo":"QiQAng/UEDVC","repo_kind":"official","path":"modules/transformer.py","file_url":"https://github.com/QiQAng/UEDVC/blob/HEAD/modules/transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"37d512fc00346438"}},{"code_sha256_prefix":"d0bf032356afceda","entry":"RegionFeatureRegression","repo":"QiQAng/UEDVC","repo_kind":"official","path":"modules/transformer.py","file_url":"https://github.com/QiQAng/UEDVC/blob/HEAD/modules/transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d0bf032356afceda"}},{"code_sha256_prefix":"354006a4908f58bd","entry":"attention","repo":"QiQAng/UEDVC","repo_kind":"official","path":"modules/transformer.py","file_url":"https://github.com/QiQAng/UEDVC/blob/HEAD/modules/transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"354006a4908f58bd"}},{"code_sha256_prefix":"8c106eab341fc5d3","entry":"gelu","repo":"QiQAng/UEDVC","repo_kind":"official","path":"modules/transformer.py","file_url":"https://github.com/QiQAng/UEDVC/blob/HEAD/modules/transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8c106eab341fc5d3"}},{"code_sha256_prefix":"7005f3fee84ccb6c","entry":"CrossEncoder","repo":"QiQAng/UEDVC","repo_kind":"official","path":"modules/transformer.py","file_url":"https://github.com/QiQAng/UEDVC/blob/HEAD/modules/transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7005f3fee84ccb6c"}},{"code_sha256_prefix":"39d02e6420121420","entry":"Encoder","repo":"QiQAng/UEDVC","repo_kind":"official","path":"modules/transformer.py","file_url":"https://github.com/QiQAng/UEDVC/blob/HEAD/modules/transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"39d02e6420121420"}},{"code_sha256_prefix":"398df3796e4acf81","entry":"EncoderLayer","repo":"QiQAng/UEDVC","repo_kind":"official","path":"modules/transformer.py","file_url":"https://github.com/QiQAng/UEDVC/blob/HEAD/modules/transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"398df3796e4acf81"}},{"code_sha256_prefix":"9ae51dc146c8a544","entry":"EntEncoder","repo":"QiQAng/UEDVC","repo_kind":"official","path":"modules/transformer.py","file_url":"https://github.com/QiQAng/UEDVC/blob/HEAD/modules/transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9ae51dc146c8a544"}},{"code_sha256_prefix":"c0f22ddc77e0c3f4","entry":"Transformer","repo":"QiQAng/UEDVC","repo_kind":"official","path":"modules/transformer.py","file_url":"https://github.com/QiQAng/UEDVC/blob/HEAD/modules/transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c0f22ddc77e0c3f4"}},{"code_sha256_prefix":"c810df21e2565eea","entry":"VISEncoder","repo":"QiQAng/UEDVC","repo_kind":"official","path":"modules/transformer.py","file_url":"https://github.com/QiQAng/UEDVC/blob/HEAD/modules/transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c810df21e2565eea"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}