Papers › Vote2Cap-DETR++: Decoupling Localization and Describing for End-to-End 3D Dense Captioning
Vote2Cap-DETR++: Decoupling Localization and Describing for End-to-End 3D Dense Captioning
Sijin Chen, Hongyuan Zhu, Mingsheng Li, Xin Chen, Peng Guo, Yinjie Lei, Gang Yu, Taihao Li, Tao Chen
3D dense captioning requires a model to translate its understanding of an input 3D scene into several captions associated with different object regions. Existing methods adopt a sophisticated "detect-then-describe" pipeline, which builds explicit relation modules upon a 3D detector with numerous hand-crafted components. While these methods have achieved initial success, the cascade pipeline tends to accumulate errors because of duplicated and inaccurate box estimations and messy 3D scenes. In this paper, we first propose Vote2Cap-DETR, a simple-yet-effective transformer framework that decouples the decoding process of caption generation and object localization through parallel decoding. Moreover, we argue that object localization and description generation require different levels of scene understanding, which could be challenging for a shared set of queries to capture. To this end, we propose an advanced version, Vote2Cap-DETR++, which decouples the queries into localization and caption queries to capture task-specific features. Additionally, we introduce the iterative spatial refinement strategy to vote queries for faster convergence and better localization performance. We also insert additional spatial information to the caption head for more accurate descriptions. Without bells and whistles, extensive experiments on two commonly used datasets, ScanRefer and Nr3D, demonstrate Vote2Cap-DETR and Vote2Cap-DETR++ surpass conventional "detect-then-describe" methods by a large margin. Codes will be made available at https://github.com/ch3cook-fdu/Vote2Cap-DETR.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| 3D dense captioning | Nr3D | Vote2Cap-DETR++ | BLEU-4 | 27.70 | #3 of 10 | Archive leaderboard | report |
| 3D dense captioning | Nr3D | Vote2Cap-DETR++ | CIDEr | 47.08 | #3 of 10 | Archive leaderboard | report |
| 3D dense captioning | Nr3D | Vote2Cap-DETR++ | METEOR | 25.44 | #3 of 10 | Archive leaderboard | report |
| 3D dense captioning | Nr3D | Vote2Cap-DETR++ | ROUGE-L | 55.22 | #3 of 10 | Archive leaderboard | report |
| 3D dense captioning | ScanRefer Dataset | Vote2Cap-DETR++ | BLEU-4 | 41.37 | #4 of 12 | Archive leaderboard | report |
| 3D dense captioning | ScanRefer Dataset | Vote2Cap-DETR++ | CIDEr | 76.36 | #4 of 12 | Archive leaderboard | report |
| 3D dense captioning | ScanRefer Dataset | Vote2Cap-DETR++ | METEOR | 28.70 | #4 of 12 | Archive leaderboard | report |
| 3D dense captioning | ScanRefer Dataset | Vote2Cap-DETR++ | ROUGE-L | 60.00 | #4 of 12 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections