Papers › Improving Visual Object Tracking through Visual Prompting
Improving Visual Object Tracking through Visual Prompting
Shih-Fang Chen, Jun-Cheng Chen, I-Hong Jhuo, Yen-Yu Lin
Learning a discriminative model to distinguish a target from its surrounding distractors is essential to generic visual object tracking. Dynamic target representation adaptation against distractors is challenging due to the limited discriminative capabilities of prevailing trackers. We present a new visual Prompting mechanism for generic Visual Object Tracking (PiVOT) to address this issue. PiVOT proposes a prompt generation network with the pre-trained foundation model CLIP to automatically generate and refine visual prompts, enabling the transfer of foundation model knowledge for tracking. While CLIP offers broad category-level knowledge, the tracker, trained on instance-specific data, excels at recognizing unique object instances. Thus, PiVOT first compiles a visual prompt highlighting potential target locations. To transfer the knowledge of CLIP to the tracker, PiVOT leverages CLIP to refine the visual prompt based on the similarities between candidate objects and the reference templates across potential targets. Once the visual prompt is refined, it can better highlight potential target locations, thereby reducing irrelevant prompt information. With the proposed prompting mechanism, the tracker can generate improved instance-aware feature maps through the guidance of the visual prompt, thus effectively reducing distractors. The proposed method does not involve CLIP during training, thereby keeping the same training complexity and preserving the generalization capability of the pretrained foundation model. Extensive experiments across multiple benchmarks indicate that PiVOT, using the proposed prompting method can suppress distracting objects and enhance the tracker.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Visual Object Tracking | AVisT | PiVOT-L | Success Rate | 62.2 | #1 of 7 | Archive leaderboard | report |
| Visual Object Tracking | LaSOT | PiVOT-L | AUC | 73.4 | #13 of 46 | Archive leaderboard | report |
| Visual Object Tracking | LaSOT | PiVOT-L | Normalized Precision | 84.7 | #13 of 46 | Archive leaderboard | report |
| Visual Object Tracking | LaSOT | PiVOT-L | Precision | 82.1 | #13 of 46 | Archive leaderboard | report |
| Visual Object Tracking | NeedForSpeed | PiVOT-L | AUC | 0.682 | #3 of 10 | Archive leaderboard | report |
| Visual Object Tracking | OTB-2015 | PiVOT-L | AUC | 0.712 | #6 of 18 | Archive leaderboard | report |
| Visual Object Tracking | OTB-2015 | PiVOT-L | Precision | 0.946 | #6 of 18 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections