Highlight Detection

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

4 leaderboard tables shown for this task, 4 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

2 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Parent tasks archive 2025-07-28

Most implemented papers archive 2025-07-28

30 shown of 41 papers with code (78 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• QVHighlights: Detecting Moments and Highlights in Videos via Natural Language Queries
  20 Jul 2021 4 repositories listed Syntology ran 2 of 2 samples · 0 unverified · 1 pointer-only (licence)
  Each video in the dataset is annotated with: (1) a human-written free-form NL query, (2) relevant moments in the video w.
• UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detection
  23 Mar 2022 3 repositories listed
  Finding relevant moments and highlights in videos according to natural language queries is a natural and highly valuable common need in the current video content explosion era.
• TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding
  4 Dec 2023 2 repositories listed Syntology ran 7 of 11 samples · 4 unverified
  This work proposes TimeChat, a time-sensitive multimodal large language model specifically designed for long video understanding.
• Correlation-Guided Query-Dependency Calibration for Video Temporal Grounding
  15 Nov 2023 2 repositories listed Syntology ran 2 of 11 samples · 9 unverified · 11 pointer-only (licence)
  Dummy tokens conditioned by text query take portions of the attention weights, preventing irrelevant video clips from being represented by the text query.
• Rhapsody: A Dataset for Highlight Detection in Podcasts
  26 May 2025 1 repository listed
  Podcasts have become daily companions for half a billion users.
• TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action
  2 May 2025 1 repository listed
  We propose TEMPURA (Temporal Event Masked Prediction and Understanding for Reasoning in Action), a two-stage training framework that enhances video temporal understanding.
• LD-DETR: Loop Decoder DEtection TRansformer for Video Moment Retrieval and Highlight Detection
  18 Jan 2025 1 repository listed
  Existing models usually first use contrastive learning methods to align video and text features, then fuse and extract multimodal information, and finally use a Transformer Decoder to decode multimodal information.
• Watch Video, Catch Keyword: Context-aware Keyword Attention for Moment Retrieval and Highlight Detection
  5 Jan 2025 1 repository listed Syntology ran 8 of 10 samples · 2 unverified · 10 pointer-only (licence)
  In this paper, we present a novel Video Context-aware Keyword Attention module that overcomes this limitation by capturing keyword variation within the context of the entire video.
• FlashVTG: Feature Layering and Adaptive Score Handling Network for Video Temporal Grounding
  18 Dec 2024 1 repository listed Syntology ran 2 of 13 samples · 11 unverified · 13 pointer-only (licence)
  For short-moment retrieval, FlashVTG increases mAP to 125% of previous SOTA performance.
• Video Repurposing from User Generated Content: A Large-scale Dataset and Benchmark
  12 Dec 2024 1 repository listed Syntology ran 0 of 11 samples · 11 unverified
  The demand for producing short-form videos for sharing on social media platforms has experienced significant growth in recent times.
• VideoLights: Feature Refinement and Cross-Task Alignment Transformer for Joint Video Highlight Detection and Moment Retrieval
  2 Dec 2024 1 repository listed
  Video Highlight Detection and Moment Retrieval (HD/MR) are essential in video analysis.
• VideoLLM Knows When to Speak: Enhancing Time-Sensitive Video Comprehension with Video-Text Duet Interaction Format
  27 Nov 2024 1 repository listed Syntology ran 0 of 2 samples · 2 unverified
  We construct MMDuetIT, a video-text training dataset designed to adapt VideoLLMs to video-text duet interaction format.
• Number it: Temporal Grounding Videos like Flipping Manga
  15 Nov 2024 1 repository listed Syntology ran 1 of 11 samples · 10 unverified
  Video Large Language Models (Vid-LLMs) have made remarkable advancements in comprehending video content for QA dialogue.
• TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning
  25 Oct 2024 1 repository listed Syntology ran 3 of 6 samples · 3 unverified
  This paper proposes TimeSuite, a collection of new designs to adapt the existing short-form video MLLMs for long video understanding, including a simple yet efficient framework to process long video sequence, a…
• Saliency-Guided DETR for Moment Retrieval and Highlight Detection
  2 Oct 2024 1 repository listed
  Combined with the introduced Saliency-Guided Cross Attention mechanism and a hybrid DETR architecture, our approach significantly enhances performance in both moment retrieval and highlight detection tasks.
• Lighthouse: A User-Friendly Library for Reproducible Video Moment Retrieval and Highlight Detection
  6 Aug 2024 1 repository listed Syntology ran 5 of 11 samples · 6 unverified
  Lighthouse addresses these issues by implementing a unified reproducible codebase that includes six models, three features, and five datasets.
• Prior Knowledge Integration via LLM Encoding and Pseudo Event Regulation for Video Moment Retrieval
  21 Jul 2024 1 repository listed Syntology ran 5 of 11 samples · 6 unverified
  Through a feasibility study, we demonstrate that LLM encoders effectively refine inter-concept relations in multimodal embeddings, even without being trained on textual embeddings.
• VTG-LLM: Integrating Timestamp Knowledge into Video LLMs for Enhanced Video Temporal Grounding
  22 May 2024 1 repository listed Syntology ran 13 of 17 samples · 4 unverified
  Video Temporal Grounding (VTG) strives to accurately pinpoint event timestamps in a specific video using linguistic queries, significantly impacting downstream tasks like video browsing and editing.
• Task-Driven Exploration: Decoupling and Inter-Task Feedback for Joint Moment Retrieval and Highlight Detection
  14 Apr 2024 1 repository listed Syntology ran 6 of 10 samples · 4 unverified · 10 pointer-only (licence)
  Video moment retrieval and highlight detection are two highly valuable tasks in video understanding, but until recently they have been jointly studied.
• R^2-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Grounding
  2 Apr 2024 1 repository listed
  Video temporal grounding (VTG) is a fine-grained video understanding problem that aims to ground relevant clips in untrimmed videos given natural language queries.
• Unleash the Potential of CLIP for Video Highlight Detection
  2 Apr 2024 1 repository listed Syntology ran 3 of 7 samples · 4 unverified
  Multimodal and large language models (LLMs) have revolutionized the utilization of open-world knowledge, unlocking novel potentials across various tasks and applications.
• R²-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Grounding
  31 Mar 2024 1 repository listed
  Video temporal grounding (VTG) is a fine-grained video understanding problem that aims to ground relevant clips in untrimmed videos given natural language queries.
• TR-DETR: Task-Reciprocal Transformer for Joint Moment Retrieval and Highlight Detection
  4 Jan 2024 1 repository listed Syntology ran 11 of 17 samples · 6 unverified · 17 pointer-only (licence)
  Finally, a task cooperation module is constructed to refine the retrieval pipeline and the highlight score prediction process by utilizing the reciprocity between MR and HD.
• Bridging the Gap: A Unified Video Comprehension Framework for Moment Retrieval and Highlight Detection
  28 Nov 2023 1 repository listed Syntology ran 9 of 12 samples · 3 unverified
  Video Moment Retrieval (MR) and Highlight Detection (HD) have attracted significant attention due to the growing demand for video analysis.
• Joint network for specular highlight detection and adversarial generation of specular-free images trained with polarimetric data
  28 Nov 2023 1 repository listed
  Once trained, SHMGAN is able to generate specular-free images from a single RGB image as input; without requiring any additional external labels.
• UniVTG: Towards Unified Video-Language Temporal Grounding
  31 Jul 2023 1 repository listed Syntology ran 10 of 16 samples · 6 unverified
  Most methods in this direction develop taskspecific models that are trained with type-specific labels, such as moment retrieval (time interval) and highlight detection (worthiness curve), which limits their abilities to…
• Joint Moment Retrieval and Highlight Detection Via Natural Language Queries
  8 May 2023 1 repository listed
  Video summarization has become an increasingly important task in the field of computer vision due to the vast amount of video content available on the internet.
• MH-DETR: Video Moment and Highlight Detection with Cross-modal Transformer
  29 Apr 2023 1 repository listed
  With the increasing demand for video understanding, video moment and highlight detection (MHD) has emerged as a critical research topic.
• Collaborative Noisy Label Cleaner: Learning Scene-aware Trailers for Multi-modal Highlight Detection in Movies
  26 Mar 2023 1 repository listed
  Based on existing efforts, this work has two observations: (1) For different annotators, labeling highlight has uncertainty, which leads to inaccurate and time-consuming annotations.
• Query-Dependent Video Representation for Moment Retrieval and Highlight Detection
  24 Mar 2023 1 repository listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  As we observe the insignificant role of a given query in transformer architectures, our encoding module starts with cross-attention layers to explicitly inject the context of text query into video representation.

Syntology lines on 18 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.