{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/2408-00969","title":"Visible-Thermal Multiple Object Tracking: Large-scale Video Dataset and Progressive Fusion Approach","arxiv_id":"2408.00969","date":"2024-08-02","proceeding":null,"authors":["Yabin Zhu","Qianwu Wang","Chenglong Li","Jin Tang","Zhixiang Huang"],"abstract":"The complementary benefits from visible and thermal infrared data are widely utilized in various computer vision task, such as visual tracking, semantic segmentation and object detection, but rarely explored in Multiple Object Tracking (MOT). In this work, we contribute a large-scale Visible-Thermal video benchmark for MOT, called VT-MOT. VT-MOT has the following main advantages. 1) The data is large scale and high diversity. VT-MOT includes 582 video sequence pairs, 401k frame pairs from surveillance, drone, and handheld platforms. 2) The cross-modal alignment is highly accurate. We invite several professionals to perform both spatial and temporal alignment frame by frame. 3) The annotation is dense and high-quality. VT-MOT has 3.99 million annotation boxes annotated and double-checked by professionals, including heavy occlusion and object re-acquisition (object disappear and reappear) challenges. To provide a strong baseline, we design a simple yet effective tracking framework, which effectively fuses temporal information and complementary information of two modalities in a progressive manner, for robust visible-thermal MOT. A comprehensive experiment are conducted on VT-MOT and the results prove the superiority and effectiveness of the proposed method compared with state-of-the-art methods. From the evaluation results and analysis, we specify several potential future directions for visible-thermal MOT. The project is released in https://github.com/wqw123wqw/PFTrack.","url_abs":"https://arxiv.org/abs/2408.00969v1","url_pdf":"https://arxiv.org/pdf/2408.00969v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"2408-00969","repo_url":"https://github.com/wqw123wqw/pftrack","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"multiple-object-tracking","task_name":"Multiple Object Tracking"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"object-tracking","task_name":"Object Tracking"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"visual-tracking","task_name":"Visual Tracking"},{"task_slug":"cross-modal-alignment","task_name":"cross-modal alignment"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2408.00969","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2408.00969"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wqw123wqw/pftrack","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3,"unverified":4},"by_repo_kind":{"official":{"samples":7,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0ce058e665602da2","entry":"init_config","repo":"wqw123wqw/pftrack","repo_kind":"official","path":"trackeval/utils.py","file_url":"https://github.com/wqw123wqw/pftrack/blob/HEAD/trackeval/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0ce058e665602da2"}},{"code_sha256_prefix":"f33c76e72d8a6635","entry":"load_multiple_tracker_summaries","repo":"wqw123wqw/pftrack","repo_kind":"official","path":"trackeval/plotting.py","file_url":"https://github.com/wqw123wqw/pftrack/blob/HEAD/trackeval/plotting.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f33c76e72d8a6635"}},{"code_sha256_prefix":"5ef3f617efb9ddc4","entry":"update_config","repo":"wqw123wqw/pftrack","repo_kind":"official","path":"trackeval/utils.py","file_url":"https://github.com/wqw123wqw/pftrack/blob/HEAD/trackeval/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5ef3f617efb9ddc4"}},{"code_sha256_prefix":"06d2be4ac9d0e95b","entry":"geometric_mean","repo":"wqw123wqw/pftrack","repo_kind":"official","path":"trackeval/plotting.py","file_url":"https://github.com/wqw123wqw/pftrack/blob/HEAD/trackeval/plotting.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"06d2be4ac9d0e95b"}},{"code_sha256_prefix":"00c662d6d9709fe4","entry":"jaccard","repo":"wqw123wqw/pftrack","repo_kind":"official","path":"trackeval/plotting.py","file_url":"https://github.com/wqw123wqw/pftrack/blob/HEAD/trackeval/plotting.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"00c662d6d9709fe4"}},{"code_sha256_prefix":"df1bd31f4da1bed6","entry":"time","repo":"wqw123wqw/pftrack","repo_kind":"official","path":"trackeval/_timing.py","file_url":"https://github.com/wqw123wqw/pftrack/blob/HEAD/trackeval/_timing.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"df1bd31f4da1bed6"}},{"code_sha256_prefix":"9792bf1150d36891","entry":"validate_metrics_list","repo":"wqw123wqw/pftrack","repo_kind":"official","path":"trackeval/utils.py","file_url":"https://github.com/wqw123wqw/pftrack/blob/HEAD/trackeval/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9792bf1150d36891"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}