Dense Captioning

Benchmarks archive 2025-07-28

1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

2 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

1 subtask in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 34 papers with code (69 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• 3D-LLM: Injecting the 3D World into Large Language Models
  24 Jul 2023 5 repositories listed Syntology ran 5 of 10 samples · 5 unverified · 10 pointer-only (licence)
  Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs.
• Dense-Captioning Events in Videos
  2 May 2017 4 repositories listed
  We also introduce ActivityNet Captions, a large-scale benchmark for dense-captioning events.
• A Hierarchical Approach for Generating Descriptive Image Paragraphs
  20 Nov 2016 3 repositories listed Syntology ran 0 of 9 samples · 9 unverified
  Recent progress on image captioning has made it possible to generate novel sentences describing images in natural language, but compressing an image into a single sentence can describe visual content in only coarse…
• ComiCap: A VLMs pipeline for dense captioning of Comic Panels
  24 Sep 2024 2 repositories listed
  The comic domain is rapidly advancing with the development of single- and multi-page analysis and synthesis models.
• TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding
  4 Dec 2023 2 repositories listed Syntology ran 7 of 11 samples · 4 unverified
  This work proposes TimeChat, a time-sensitive multimodal large language model specifically designed for long video understanding.
• STSBench: A Spatio-temporal Scenario Benchmark for Multi-modal Large Language Models in Autonomous Driving
  6 Jun 2025 1 repository listed
  Applied to the NuScenes dataset, we present STSnu, the first benchmark that evaluates the spatio-temporal reasoning capabilities of VLMs based on comprehensive 3D perception.
• TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action
  2 May 2025 1 repository listed
  We propose TEMPURA (Temporal Event Masked Prediction and Understanding for Reasoning in Action), a two-stage training framework that enhances video temporal understanding.
• PerLA: Perceptive 3D Language Assistant
  29 Nov 2024 1 repository listed
  Enabling Large Language Models (LLMs) to understand the 3D physical world is an emerging yet challenging research direction.
• PaveCap: The First Multimodal Framework for Comprehensive Pavement Condition Assessment with Dense Captioning and PCI Estimation
  7 Aug 2024 1 repository listed
  The framework consists of two main parts: a Single-Shot PCI Estimation Network and a Dense Captioning Network.
• 3D Vision and Language Pretraining with Large-Scale Synthetic Data
  8 Jul 2024 1 repository listed Syntology ran 10 of 10 samples · 0 unverified
  3D Vision-Language Pre-training (3D-VLP) aims to provide a pre-train model which can bridge 3D scenes with natural language, which is an important technique for embodied intelligence.
• Details Make a Difference: Object State-Sensitive Neurorobotic Task Planning
  14 Jun 2024 1 repository listed Syntology ran 5 of 7 samples · 2 unverified
  The state of an object reflects its current status or condition and is important for a robot's task planning and manipulation.
• Grounded 3D-LLM with Referent Tokens
  16 May 2024 1 repository listed Syntology ran 9 of 17 samples · 8 unverified · 17 pointer-only (licence)
  To facilitate the use of referent tokens in subsequent language modeling, we provide a large-scale, automatically curated grounded scene-text dataset with over 1 million phrase-to-region correspondences and introduce…
• PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning
  25 Apr 2024 1 repository listed Syntology ran 0 of 2 samples · 2 unverified · 2 pointer-only (licence)
  PLLaVA achieves new state-of-the-art performance on modern benchmark datasets for both video question-answer and captioning tasks.
• Rethinking 3D Dense Caption and Visual Grounding in A Unified Framework through Prompt-based Localization
  17 Apr 2024 1 repository listed
  3D Visual Grounding (3DVG) and 3D Dense Captioning (3DDC) are two crucial tasks in various 3D applications, which require both shared and complementary information in localization and visual-language relationships.
• TOD3Cap: Towards 3D Dense Captioning in Outdoor Scenes
  28 Mar 2024 1 repository listed Syntology ran 10 of 16 samples · 6 unverified · 16 pointer-only (licence)
  However, the exploration of 3D dense captioning in outdoor scenes is hindered by two major challenges: 1) the domain gap between indoor and outdoor scenes, such as dynamics and sparse visual inputs, makes it difficult…
• ControlCap: Controllable Region-level Captioning
  31 Jan 2024 1 repository listed Syntology ran 3 of 4 samples · 1 unverified · 4 pointer-only (licence)
  The multimodal model is constrained to generate captions within a few sub-spaces containing the control words, which increases the opportunity of hitting less frequent captions, alleviating the caption degeneration…
• LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding Reasoning and Planning
  1 Jan 2024 1 repository listed
  Recent progress in Large Multimodal Models (LMM) has opened up great possibilities for various applications in the field of human-machine interactions.
• LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning
  30 Nov 2023 1 repository listed Syntology ran 6 of 6 samples · 0 unverified
  However, developing LMMs that can comprehend, reason, and plan in complex and diverse 3D environments remains a challenging topic, especially considering the demand for understanding permutation-invariant point cloud 3D…
• Vote2Cap-DETR++: Decoupling Localization and Describing for End-to-End 3D Dense Captioning
  6 Sep 2023 1 repository listed
  Moreover, we argue that object localization and description generation require different levels of scene understanding, which could be challenging for a shared set of queries to capture.
• 3D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignment
  8 Aug 2023 1 repository listed Syntology ran 4 of 6 samples · 2 unverified
  3D vision-language grounding (3D-VL) is an emerging field that aims to connect the 3D physical world with natural language, which is crucial for achieving embodied intelligence.
• Enhancing Vision-Language Pre-Training with Jointly Learned Questioner and Dense Captioner
  19 May 2023 1 repository listed
  In this paper, we propose a novel method called Joint QA and DC GEneration (JADE), which utilizes a pre-trained multimodal model and easily-crawled image-text pairs to automatically generate and filter large-scale VQA…
• IIITD-20K: Dense captioning for Text-Image ReID
  8 May 2023 1 repository listed
  IIITD-20K comprises of 20, 000 unique identities captured in the wild and provides a rich dataset for text-to-image ReID.
• End-to-End 3D Dense Captioning with Vote2Cap-DETR
  6 Jan 2023 1 repository listed Syntology ran 1 of 3 samples · 2 unverified
  Compared with prior arts, our framework has several appealing advantages: 1) Without resorting to numerous hand-crafted components, our method is based on a full transformer encoder-decoder architecture with a learnable…
• Context-Aware Alignment and Mutual Masking for 3D-Language Pre-Training
  1 Jan 2023 1 repository listed
  The current approaches for 3D visual reasoning are task-specific, and lack pre-training methods to learn generic representations that can transfer across various tasks.
• GRiT: A Generative Region-to-text Transformer for Object Understanding
  1 Dec 2022 1 repository listed Syntology ran 5 of 8 samples · 3 unverified · 8 pointer-only (licence)
  Specifically, GRiT consists of a visual encoder to extract image features, a foreground object extractor to localize objects, and a text decoder to generate open-set object descriptions.
• Spatiality-guided Transformer for 3D Dense Captioning on Point Clouds
  22 Apr 2022 1 repository listed
  Dense captioning in 3D point clouds is an emerging vision-and-language task involving object-level 3D scene understanding.
• MORE: Multi-Order RElation Mining for Dense Captioning in 3D Scenes
  10 Mar 2022 1 repository listed Syntology ran 2 of 9 samples · 7 unverified
  3D dense captioning is a recently-proposed novel task, where point clouds contain more geometric information than the 2D counterpart.
• X-Trans2Cap: Cross-Modal Knowledge Transfer using Transformer for 3D Dense Captioning
  2 Mar 2022 1 repository listed
  Thus, a more faithful caption can be generated only using point clouds during the inference.
• Integrating Visuospatial, Linguistic, and Commonsense Structure into Story Visualization
  1 Nov 2021 1 repository listed
  Such information is even more important for story visualization since its inputs have an explicit narrative structure that needs to be translated into an image sequence (or visual story).
• Integrating Visuospatial, Linguistic and Commonsense Structure into Story Visualization
  21 Oct 2021 1 repository listed
  Prior work in this domain has shown that there is ample room for improvement in the generated image sequence in terms of visual quality, consistency and relevance.

Syntology lines on 14 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.