{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2508-17817","title":"TemCoCo: Temporally Consistent Multi-modal Video Fusion with Visual-Semantic Collaboration","arxiv_id":"2508.17817","date":"2025-08-25","proceeding":"ICCV","authors":["Meiqi Gong","Hao Zhang","Xunpeng Yi","Linfeng Tang","Jiayi Ma"],"abstract":"Existing multi-modal fusion methods typically apply static frame-based image fusion techniques directly to video fusion tasks, neglecting inherent temporal dependencies and leading to inconsistent results across frames. To address this limitation, we propose the first video fusion framework that explicitly incorporates temporal modeling with visual-semantic collaboration to simultaneously ensure visual fidelity, semantic accuracy, and temporal consistency. First, we introduce a visual-semantic interaction module consisting of a semantic branch and a visual branch, with Dinov2 and VGG19 employed for targeted distillation, allowing simultaneous enhancement of both the visual and semantic representations. Second, we pioneer integrate the video degradation enhancement task into the video fusion pipeline by constructing a temporal cooperative module, which leverages temporal dependencies to facilitate weak information recovery. Third, to ensure temporal consistency, we embed a temporal-enhanced mechanism into the network and devise a temporal loss to guide the optimization process. Finally, we introduce two innovative evaluation metrics tailored for video fusion, aimed at assessing the temporal consistency of the generated fused videos. Extensive experimental results on public video datasets demonstrate the superiority of our method. Our code is released at https://github.com/Meiqi-Gong/TemCoCo.","url_abs":"https://arxiv.org/abs/2508.17817","url_pdf":"https://arxiv.org/pdf/2508.17817","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2508.17817","atlas_url":"https://app.syntology.ai/?focus=2508.17817","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2508.17817"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/Meiqi-Gong/TemCoCo","reach":{"status":"ok"}},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/Linfeng-Tang/M3SVD","reach":{"status":"ok","spdx":"MIT"}},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/xiehousheng/HDO","reach":{"status":"ok"}}],"summary":{"ran_honours":1,"ran_draft_wrong":2,"unverified":11},"by_repo_kind":{"found_in_text":{"samples":14,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":14,"samples":[{"code_sha256_prefix":"031b02fb990284ba","entry":"box_xyxy_to_cxcywh","repo":"Meiqi-Gong/TemCoCo","repo_kind":"found_in_text","path":"utils.py","file_url":"https://github.com/Meiqi-Gong/TemCoCo/blob/HEAD/utils.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"031b02fb990284ba"}},{"code_sha256_prefix":"9bed061f167dd318","entry":"conv1x1","repo":"Meiqi-Gong/TemCoCo","repo_kind":"found_in_text","path":"archs/net_basics.py","file_url":"https://github.com/Meiqi-Gong/TemCoCo/blob/HEAD/archs/net_basics.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9bed061f167dd318"}},{"code_sha256_prefix":"f3d374db4177f20c","entry":"conv3x3","repo":"Meiqi-Gong/TemCoCo","repo_kind":"found_in_text","path":"archs/net_basics.py","file_url":"https://github.com/Meiqi-Gong/TemCoCo/blob/HEAD/archs/net_basics.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f3d374db4177f20c"}},{"code_sha256_prefix":"a361b215305fa3a4","entry":"bilinear_sampling","repo":"Meiqi-Gong/TemCoCo","repo_kind":"found_in_text","path":"SEA_RAFT/core/corr.py","file_url":"https://github.com/Meiqi-Gong/TemCoCo/blob/HEAD/SEA_RAFT/core/corr.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a361b215305fa3a4"}},{"code_sha256_prefix":"3acc39c2c638ec21","entry":"build_dataloader","repo":"Meiqi-Gong/TemCoCo","repo_kind":"found_in_text","path":"dataset.py","file_url":"https://github.com/Meiqi-Gong/TemCoCo/blob/HEAD/dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3acc39c2c638ec21"}},{"code_sha256_prefix":"3c6e3a4cc099898b","entry":"conv3x3_3d","repo":"Meiqi-Gong/TemCoCo","repo_kind":"found_in_text","path":"archs/net_basics.py","file_url":"https://github.com/Meiqi-Gong/TemCoCo/blob/HEAD/archs/net_basics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3c6e3a4cc099898b"}},{"code_sha256_prefix":"83506963d4c65ed7","entry":"coords_corr","repo":"Meiqi-Gong/TemCoCo","repo_kind":"found_in_text","path":"SEA_RAFT/core/corr.py","file_url":"https://github.com/Meiqi-Gong/TemCoCo/blob/HEAD/SEA_RAFT/core/corr.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"83506963d4c65ed7"}},{"code_sha256_prefix":"27a014ca482ec5af","entry":"coords_feature","repo":"Meiqi-Gong/TemCoCo","repo_kind":"found_in_text","path":"SEA_RAFT/core/corr.py","file_url":"https://github.com/Meiqi-Gong/TemCoCo/blob/HEAD/SEA_RAFT/core/corr.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"27a014ca482ec5af"}},{"code_sha256_prefix":"45b6edb2cf78cda2","entry":"create_window","repo":"Meiqi-Gong/TemCoCo","repo_kind":"found_in_text","path":"loss.py","file_url":"https://github.com/Meiqi-Gong/TemCoCo/blob/HEAD/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"45b6edb2cf78cda2"}},{"code_sha256_prefix":"3227d1680c9a7d97","entry":"feaCD","repo":"Meiqi-Gong/TemCoCo","repo_kind":"found_in_text","path":"metrics.py","file_url":"https://github.com/Meiqi-Gong/TemCoCo/blob/HEAD/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3227d1680c9a7d97"}},{"code_sha256_prefix":"e2424ce033ba3a0f","entry":"gaussian","repo":"Meiqi-Gong/TemCoCo","repo_kind":"found_in_text","path":"loss.py","file_url":"https://github.com/Meiqi-Gong/TemCoCo/blob/HEAD/loss.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e2424ce033ba3a0f"}},{"code_sha256_prefix":"22e4e633221fd8e0","entry":"imfrombytes","repo":"Meiqi-Gong/TemCoCo","repo_kind":"found_in_text","path":"utils.py","file_url":"https://github.com/Meiqi-Gong/TemCoCo/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"22e4e633221fd8e0"}},{"code_sha256_prefix":"f6a9d77459cab804","entry":"resize","repo":"Meiqi-Gong/TemCoCo","repo_kind":"found_in_text","path":"utils.py","file_url":"https://github.com/Meiqi-Gong/TemCoCo/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f6a9d77459cab804"}},{"code_sha256_prefix":"48b19007515fd83b","entry":"ssim","repo":"Meiqi-Gong/TemCoCo","repo_kind":"found_in_text","path":"loss.py","file_url":"https://github.com/Meiqi-Gong/TemCoCo/blob/HEAD/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"48b19007515fd83b"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CV","source":"arxiv_api"},"syntology_extracted_results":{"kind":"leaderboard_placements","source":"Syntology's leaderboard-shaped extractor over the paper's own arXiv-HTML tables: a model pointed at a cell, the number was read from that cell and checked against the board's metric, dataset, split and scale, and an independent check accepted the entry; not reviewed by the paper's authors or the archive's editors","extractor_model":"global.anthropic.claude-sonnet-4-5-20250929-v1:0","verifier_model":null,"prompt_sha":"fa63d4bb9d755694","coverage":{"sentence":"Syntology has checked 6,885 of the 9,623 papers on this site that are newer than the archive; results from the others appear after they are checked.","papers_newer_than_archive":9623,"papers_checked":6885},"entries":[],"not_placed":{"boards":0,"rejected_by_independent_check":0,"refused_by_a_rule":0,"check_did_not_answer":0,"proposed_without_a_cell":0,"declined_by_site":0}}}