{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/associate-everything-detected-facilitating","title":"Associate Everything Detected: Facilitating Tracking-by-Detection to the Unknown","arxiv_id":"2409.09293","date":"2024-09-14","proceeding":null,"authors":["Zimeng Fang","Chao Liang","Xue Zhou","Shuyuan Zhu","Xi Li"],"abstract":"Multi-object tracking (MOT) emerges as a pivotal and highly promising branch in the field of computer vision. Classical closed-vocabulary MOT (CV-MOT) methods aim to track objects of predefined categories. Recently, some open-vocabulary MOT (OV-MOT) methods have successfully addressed the problem of tracking unknown categories. However, we found that the CV-MOT and OV-MOT methods each struggle to excel in the tasks of the other. In this paper, we present a unified framework, Associate Everything Detected (AED), that simultaneously tackles CV-MOT and OV-MOT by integrating with any off-the-shelf detector and supports unknown categories. Different from existing tracking-by-detection MOT methods, AED gets rid of prior knowledge (e.g. motion cues) and relies solely on highly robust feature learning to handle complex trajectories in OV-MOT tasks while keeping excellent performance in CV-MOT tasks. Specifically, we model the association task as a similarity decoding problem and propose a sim-decoder with an association-centric learning mechanism. The sim-decoder calculates similarities in three aspects: spatial, temporal, and cross-clip. Subsequently, association-centric learning leverages these threefold similarities to ensure that the extracted features are appropriate for continuous tracking and robust enough to generalize to unknown categories. Compared with existing powerful OV-MOT and CV-MOT methods, AED achieves superior performance on TAO, SportsMOT, and DanceTrack without any prior knowledge. Our code is available at https://github.com/balabooooo/AED.","url_abs":"https://arxiv.org/abs/2409.09293v1","url_pdf":"https://arxiv.org/pdf/2409.09293v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"associate-everything-detected-facilitating","repo_url":"https://github.com/balabooooo/aed","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"multi-object-tracking","task_name":"Multi-Object Tracking"},{"task_slug":"multiple-object-tracking","task_name":"Multiple Object Tracking"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-tracking","task_name":"Object Tracking"},{"task_slug":"video-object-tracking","task_name":"Video Object Tracking"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/multi-object-tracking-on-dancetrack","task":"Multi-Object Tracking","dataset":"DanceTrack","model":"AED","rank_in_archive_order":11,"of":37,"metrics":{"AssA":"54.3","DetA":"82.0","HOTA":"66.6","IDF1":"69.7","MOTA":"92.2"},"uses_additional_data":true},{"leaderboard":"/sota/multi-object-tracking-on-sportsmot","task":"Multi-Object Tracking","dataset":"SportsMOT","model":"AED","rank_in_archive_order":5,"of":22,"metrics":{"AssA":"70.1","DetA":"89.4","HOTA":"79.1","IDF1":"81.8","MOTA":"97.1"},"uses_additional_data":true},{"leaderboard":"/sota/multi-object-tracking-on-tao","task":"Multi-Object Tracking","dataset":"TAO","model":"AED (Co-DETR)","rank_in_archive_order":1,"of":9,"metrics":{"AssocA":"52.4","ClsA":"41.7","LocA":"71.8","TETA":"55.3"},"uses_additional_data":true},{"leaderboard":"/sota/multi-object-tracking-on-tao","task":"Multi-Object Tracking","dataset":"TAO","model":"AED (RegionCLIP)","rank_in_archive_order":5,"of":9,"metrics":{"AssocA":"38.1","ClsA":"16.2","LocA":"56.7","TETA":"37.0"},"uses_additional_data":true},{"leaderboard":"/sota/multiple-object-tracking-on-sportsmot","task":"Multiple Object Tracking","dataset":"SportsMOT","model":"AED","rank_in_archive_order":3,"of":19,"metrics":{"AssA":"70.1","DetA":"89.4","HOTA":"79.1","IDF1":"81.8","MOTA":"97.1"},"uses_additional_data":true}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2409.09293","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}