Browse State-of-the-Art › Video Description
Video Description
34 papers with code · 0 benchmarks · 9 datasets archive 2025-07-28
The goal of automatic Video Description is to tell a story about events happening in a video. While early Video Description methods produced captions for short clips that were manually segmented to contain a single event of interest, more recently dense video captioning has been proposed to both segment distinct events in time and describe them in a series of coherent sentences. This problem is a generalization of dense image region captioning and has many practical applications, such as generating textual summaries for the visually impaired, or detecting and describing important events in surveillance footage.
Source: Joint Event Detection and Description in Continuous Video Streams
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
No benchmark for this task in the archive.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
9 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 104 · with a code link: 34 · where Syntology ran a sample: 11 (8 with a run with no instrument failure, 3 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (11 of 104 tagged: 8 with a run with no instrument failure, 3 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 104, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
27 Feb 2015 5 repositories listed Syntology community repositories only · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)In this context, we propose an approach that successfully takes into account both the local and global temporal structure of videos to produce descriptions.
-
6 Apr 2019 4 repositories listedWe also introduce two tasks for video-and-language research based on VATEX: (1) Multilingual Video Captioning, aimed at describing a video in various languages with a compact unified captioning model, and (2)…
-
1 Jun 2018 4 repositories listedScene-aware dialog systems will be able to have conversations with users about the objects and events around them.
-
6 Apr 2016 3 repositories listedThis paper investigates how linguistic knowledge mined from large text corpora can aid the generation of natural language descriptions of videos.
-
17 Dec 2018 2 repositories listedOur dataset, ActivityNet-Entities, augments the challenging ActivityNet Captions dataset with 158k bounding box annotations, each grounding a noun phrase.
-
21 Jun 2018 2 repositories listed Syntology 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 3 harvested samples)We introduce a new dataset of dialogs about videos of human behaviors.
-
14 Jan 2025 1 repository listed Syntology official (archive's flag): 2 ran · 4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 4 pointer-only (licence)We introduce Tarsier2, a state-of-the-art large vision-language model (LVLM) designed for generating detailed and accurate video descriptions, while also exhibiting superior general video understanding capabilities.
-
17 Dec 2024 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 5 where Syntology's instrument failed) · 6 unverified (of 12 harvested samples)Existing methods rely on explicit alignment constraints between event locations and captions, which involve complex event proposal procedures during both training and inference.
-
11 Nov 2024 1 repository listedWe propose StoryTeller, a system for generating dense descriptions of long videos, incorporating both low-level visual concepts and high-level plot information.
-
18 Jul 2024 1 repository listedWe test the SUSTechGAN and the well-known GANs to generate driving images in adverse conditions of rain and night and apply the generated images to retrain object detection networks.
-
30 Jun 2024 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)Our second contribution is the introduction of a new benchmark -- DREAM-1K (https://tarsier-vlm.github.io/) for evaluating video description models, consisting of a new challenging dataset featuring videos from diverse…
-
27 May 2024 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 6 where Syntology's instrument failed) · 2 unverified (of 9 harvested samples) · 9 pointer-only (licence)Video Anomaly Detection (VAD) systems can autonomously monitor and identify disturbances, reducing the need for manual labor and associated costs.
-
14 Apr 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)Traffic video description and analysis have received much attention recently due to the growing demand for efficient and reliable urban surveillance systems.
-
5 Mar 2024 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 7 unverified (of 12 harvested samples) · 12 pointer-only (licence)However, the complexities of these diverse modalities pose challenges for developing an efficient multimodal emotion cause analysis (ECA) system.
-
29 Feb 2024 1 repository listedNext, we finetune a retrieval model on a small subset where the best caption of each video is manually selected and then employ the model in the whole dataset to select the best caption as the annotation.
-
26 Jun 2023 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)Surprising videos, such as funny clips, creative performances, or visual illusions, attract significant attention.
-
20 Jun 2023 1 repository listedSince the availability of the pretraining resources with Indonesian sentences is relatively limited, the applicability of those approaches to our dataset is still questionable.
-
15 May 2023 1 repository listedIn this paper, we propose a novel Video Caption Editing (VCE) task to automatically revise an existing video description guided by multi-grained user requests.
-
27 Mar 2023 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 1 where Syntology's instrument failed) · 5 unverified (of 12 harvested samples) · 1 pointer-only (licence)We explore a new task for audio-visual-language modeling called fine-grained audible video description (FAVD).
-
28 Sep 2022 1 repository listedIn video captioning, there are two kinds of hallucination: object and action hallucination.
-
12 May 2022 1 repository listedWhile there have been significant gains in the field of automated video description, the generalization performance of automated description models to novel domains remains a major barrier to using these systems in the…
-
30 Apr 2022 1 repository listedWe propose a learning based method for training a negation-aware video retrieval model.
-
22 Aug 2020 1 repository listedThis auxiliary task allows us to propose a two-stage approach to Identity-Aware Video Description.
-
18 Aug 2020 1 repository listedWith the arising concerns for the AI systems provided with direct access to abundant sensitive information, researchers seek to develop more reliable AI with implicit information sources.
-
16 Jan 2020 1 repository listed Syntology official: harvested, nothing ran · 0 ran · 2 unverified (of 2 harvested samples)Video captioning is an advanced multi-modal task which aims to describe a video clip using a natural language sentence.
-
12 Sep 2019 1 repository listedAutomatic evaluation of text generation tasks (e. g. machine translation, text summarization, image captioning and video description) usually relies heavily on task-specific metrics, such as BLEU and ROUGE.
-
13 Dec 2018 1 repository listedAmong the main issues are the fluency and coherence of the generated descriptions, and their relevance to the video.
-
5 Sep 2017 1 repository listedThis paper strives to find amidst a set of sentences the one best describing the content of a given image or video.
-
7 Apr 2017 1 repository listedWe propose a novel methodology that exploits information from temporally neighboring events, matching precisely the nature of egocentric sequences.
-
7 Nov 2016 1 repository listedWe present a method to improve video description generation by modeling higher-order interactions between video frames and described concepts.
-
A Mid-level Video Representation based on Binary Descriptors: A Case Study for Pornography Detection12 May 2016 1 repository listed
-
12 Apr 2016 1 repository listed
-
10 Apr 2016 1 repository listed
-
3 Mar 2015 1 repository listed
-
DANTE-AD: Dual-Vision Attention Network for Long-Term Audio Description31 Mar 2025 0 repositories listed
-
Cross-Modal Learning for Music-to-Music-Video Description Generation14 Mar 2025 0 repositories listed
-
VideoA11y: Method and Dataset for Accessible Video Description27 Feb 2025 0 repositories listed
-
AVD2: Accident Video Diffusion for Accident Video Description20 Feb 2025 0 repositories listed
-
Enhancing Video Understanding: Deep Neural Networks for Spatiotemporal Analysis11 Feb 2025 0 repositories listed
-
Towards Zero-Shot & Explainable Video Description by Reasoning over Graphs of Events in Space and Time14 Jan 2025 0 repositories listed
-
30 Oct 2024 0 repositories listed
-
FIOVA: A Multi-Annotator Benchmark for Human-Aligned Video Captioning20 Oct 2024 0 repositories listed
-
VideoCLIP-XL: Advancing Long Description Understanding for Video CLIP Models1 Oct 2024 0 repositories listed
-
Technical Report: Competition Solution For Modelscope-Sora24 Sep 2024 0 repositories listed
-
Kubrick: Multimodal Agent Collaborations for Synthetic Video Generation19 Aug 2024 0 repositories listed
-
https://arxiv.org/abs/2407.006342 Jul 2024 0 repositories listed
-
LLAVIDAL: A Large LAnguage VIsion Model for Daily Activities of Living13 Jun 2024 0 repositories listed
-
A Labelled Dataset for Sentiment Analysis of Videos on YouTube, TikTok, and Other Sources about the 2024 Outbreak of Measles11 Jun 2024 0 repositories listed
-
X-VARS: Introducing Explainability in Football Refereeing with Multi-Modal Large Language Model7 Apr 2024 0 repositories listed
-
Multi-modal News Understanding with Professionally Labelled Videos (ReutersViLNews)23 Jan 2024 0 repositories listed
-
ActionHub: A Large-scale Action Video Description Dataset for Zero-shot Action Recognition22 Jan 2024 0 repositories listed
-
Attention Based Encoder Decoder Model for Video Captioning in Nepali (2023)12 Dec 2023 0 repositories listed
-
Multi Sentence Description of Complex Manipulation Action Videos13 Nov 2023 0 repositories listed
-
CLearViD: Curriculum Learning for Video Description8 Nov 2023 0 repositories listed
-
Analyzing Political Figures in Real-Time: Leveraging YouTube Metadata for Sentiment Analysis28 Sep 2023 0 repositories listed
-
Synchronized Audio-Visual Frames with Fractional Positional Encoding for Transformers in Video-to-Text Translation28 Dec 2021 0 repositories listed
-
Relational Graph Learning for Grounded Video Description Generation2 Dec 2021 0 repositories listed
-
1 Dec 2021 0 repositories listed
-
NarrationBot and InfoBot: A Hybrid System for Automated Video Description7 Nov 2021 0 repositories listed
-
Visual-aware Attention Dual-stream Decoder for Video Captioning16 Oct 2021 0 repositories listed
-
Boosting Video Captioning with Dynamic Loss Network25 Jul 2021 0 repositories listed
-
Efficient data-driven encoding of scene motion using Eccentricity3 Mar 2021 0 repositories listed
-
The Role of the Input in Natural Language Video Description9 Feb 2021 0 repositories listed
-
Unbox the Blackbox: Predict and Interpret YouTube Viewership Using Deep Learning21 Dec 2020 0 repositories listed
-
MSVD-Turkish: A Comprehensive Multimodal Dataset for Integrated Vision and Language Research in Turkish13 Dec 2020 0 repositories listed
-
A Comprehensive Review on Recent Methods and Challenges of Video Description30 Nov 2020 0 repositories listed
-
Active Learning for Video Description With Cluster-Regularized Ensemble Ranking27 Jul 2020 0 repositories listed
-
Multi-Layer Content Interaction Through Quaternion Product For Visual Question Answering3 Jan 2020 0 repositories listed
-
Prediction and Description of Near-Future Activities in Video2 Aug 2019 0 repositories listed
-
End-to-End Video Captioning4 Apr 2019 0 repositories listed
-
A Dataset for Telling the Stories of Social Media Videos1 Oct 2018 0 repositories listed
-
Incorporating Background Knowledge into Video Description Generation1 Oct 2018 0 repositories listed
-
Attentive Sequence to Sequence Translation for Localizing Clips of Interest by Natural Language Descriptions27 Aug 2018 0 repositories listed
-
Bridge Video and Text with Cascade Syntactic Structure1 Aug 2018 0 repositories listed
-
Multimodal Neural Machine Translation for Low-resource Language Pairs using Synthetic Data1 Jul 2018 0 repositories listed
-
Interpretable Video Captioning via Trajectory Structured Localization1 Jun 2018 0 repositories listed
-
Video Description: A Survey of Methods, Datasets and Evaluation Metrics1 Jun 2018 0 repositories listed
-
Incorporating Semantic Attention in Video Description Generation1 May 2018 0 repositories listed
-
Integrating both Visual and Audio Cues for Enhanced Video Caption22 Nov 2017 0 repositories listed
-
Attend and Interact: Higher-Order Object Interactions for Video Understanding16 Nov 2017 0 repositories listed
-
Incorporating Global Visual Features into Attention-based Neural Machine Translation.1 Sep 2017 0 repositories listed
-
Task-Driven Dynamic Fusion: Reducing Ambiguity in Video Description1 Jul 2017 0 repositories listed
-
Attention-Based Multimodal Fusion for Video Description11 Jan 2017 0 repositories listed
-
Generating Video Description using Sequence-to-sequence Model with Temporal Attention1 Dec 2016 0 repositories listed
-
Hierarchical Boundary-Aware Neural Encoder for Video Captioning28 Nov 2016 0 repositories listed
-
Neural Headline Generation on Abstract Meaning Representation1 Nov 2016 0 repositories listed
-
Natural Language Descriptions of Human Activities Scenes: Corpus Generation and Analysis1 Aug 2016 0 repositories listed
-
SHEF-Multimodal: Grounding Machine Translation on Images1 Aug 2016 0 repositories listed
-
VideoMCC: a New Benchmark for Video Comprehension23 Jun 2016 0 repositories listed
-
Bidirectional Long-Short Term Memory for Video Description15 Jun 2016 0 repositories listed
-
1 Jun 2016 0 repositories listed
-
Noisy Parallel Approximate Decoding for Conditional Recurrent Language Model12 May 2016 0 repositories listed
-
Vectors of Locally Aggregated Centers for Compact Video Representation13 Sep 2015 0 repositories listed
-
A Multi-scale Multiple Instance Video Description Network21 May 2015 0 repositories listed
-
Probabilistic Soft Logic for Semantic Textual Similarity1 Jun 2014 0 repositories listed
-
24 Mar 2014 0 repositories listed
-
Semantic Neighborhoods as Hypergraphs1 Aug 2013 0 repositories listed
-
1 Jun 2013 0 repositories listed
-
Better Exploiting Motion for Better Action Recognition1 Jun 2013 0 repositories listed
Syntology lines on 12 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections