Papers › Tracking Every Thing in the Wild

Tracking Every Thing in the Wild

26 Jul 2022arXiv:2207.12978archive 2025-07-28

Siyuan Li, Martin Danelljan, Henghui Ding, Thomas E. Huang, Fisher Yu

Current multi-category Multiple Object Tracking (MOT) metrics use class labels to group tracking results for per-class evaluation. Similarly, MOT methods typically only associate objects with the same class predictions. These two prevalent strategies in MOT implicitly assume that the classification performance is near-perfect. However, this is far from the case in recent large-scale MOT datasets, which contain large numbers of classes with many rare or semantically similar categories. Therefore, the resulting inaccurate classification leads to sub-optimal tracking and inadequate benchmarking of trackers. We address these issues by disentangling classification from tracking. We introduce a new metric, Track Every Thing Accuracy (TETA), breaking tracking measurement into three sub-factors: localization, association, and classification, allowing comprehensive benchmarking of tracking performance even under inaccurate classification. TETA also deals with the challenging incomplete annotation problem in large-scale tracking datasets. We further introduce a Track Every Thing tracker (TETer), that performs association using Class Exemplar Matching (CEM). Our experiments show that TETA evaluates trackers more comprehensively, and TETer achieves significant improvements on the challenging large-scale datasets BDD100K and TAO compared to the state-of-the-art.

PaperPDFCode

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

SysCV/tet officialmentioned on GitHubpytorchApache-2.0 report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

BenchmarkingClassificationMulti-Object TrackingMultiple Object TrackingObject Tracking

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Multi-Object Tracking TAO TETer-HTC AssocA 37.53 #6 of 9 Archive leaderboard report
Multi-Object Tracking TAO TETer-HTC ClsA 15.70 #6 of 9 Archive leaderboard report
Multi-Object Tracking TAO TETer-HTC LocA 57.53 #6 of 9 Archive leaderboard report
Multi-Object Tracking TAO TETer-HTC TETA 36.85 #6 of 9 Archive leaderboard report
Multi-Object Tracking TAO TETer-SwinT AssocA 36.71 #7 of 9 Archive leaderboard report
Multi-Object Tracking TAO TETer-SwinT ClsA 15.03 #7 of 9 Archive leaderboard report
Multi-Object Tracking TAO TETer-SwinT LocA 52.10 #7 of 9 Archive leaderboard report
Multi-Object Tracking TAO TETer-SwinT TETA 34.61 #7 of 9 Archive leaderboard report
Multi-Object Tracking TAO TETer AssocA 35.02 #8 of 9 Archive leaderboard report
Multi-Object Tracking TAO TETer ClsA 13.16 #8 of 9 Archive leaderboard report
Multi-Object Tracking TAO TETer LocA 51.58 #8 of 9 Archive leaderboard report
Multi-Object Tracking TAO TETer TETA 33.25 #8 of 9 Archive leaderboard report
Multiple Object Tracking BDD100K val TETer AssocA 52.9 #7 of 9 Archive leaderboard report
Multiple Object Tracking BDD100K val TETer TETA 50.8 #7 of 9 Archive leaderboard report
Multiple Object Tracking BDD100K val TETer mIDF1 53.3 #7 of 9 Archive leaderboard report
Multiple Object Tracking BDD100K val TETer mMOTA 39.1 #7 of 9 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections