Browse State-of-the-Art › Highlight Detection
Highlight Detection
41 papers with code · 4 benchmarks · 2 datasets archive 2025-07-28
https://youtu.be/pJ0auP7dbcY?si=vSiZevfJ57YUKC2q
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
4 leaderboard tables shown for this task, 4 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
Syntology 2 newer papers were judged to report on this task's benchmarks so far (Claude Sonnet 4.5, reading their tables: a model's judgement, not a result; 0 placed): QVHighlights (1 judged, 0 placed), TvSum (1 judged, 0 placed). Drawn from 8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read).
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| QVHighlights (21 rows) | SG-DETR (w/ PT) | Saliency-Guided DETR for Moment Retrieval and Highlight Detection | code | — | Compare |
| TvSum (7 rows) | FlashVTG | FlashVTG: Feature Layering and Adaptive Score Handling Network for... | code | Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 2 violated, 2 with no contract checked; 5 where Syntology's instrument failed) · 4 unverified | Compare |
| YouTube Highlights (7 rows) | SG-DETR (w/ PT) | Saliency-Guided DETR for Moment Retrieval and Highlight Detection | code | — | Compare |
| arabiska (1 row) | Kenan Kanan | 100,000 Podcasts: A Spoken English Document Corpus | — | — | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
2 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 78 · with a code link: 41 · where Syntology ran a sample: 18 (17 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (18 of 78 tagged: 17 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 78 of 78, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
20 Jul 2021 4 repositories listed Syntology official (archive's flag): 1 ran · 2 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 2 samples that ran constructed an object rather than computing a result (of 2 harvested samples) · 1 pointer-only (licence)Each video in the dataset is annotated with: (1) a human-written free-form NL query, (2) relevant moments in the video w. r. t.
-
23 Mar 2022 3 repositories listedFinding relevant moments and highlights in videos according to natural language queries is a natural and highly valuable common need in the current video content explosion era.
-
4 Dec 2023 2 repositories listed Syntology community repositories only · 10 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 1 violated, 4 with no contract checked; 5 where Syntology's instrument failed) · 1 unverified (of 11 harvested samples) · 1 pointer-only (licence)This work proposes TimeChat, a time-sensitive multimodal large language model specifically designed for long video understanding.
-
15 Nov 2023 2 repositories listed Syntology official (archive's flag): 7 ran · 8 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 2 violated, 1 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified (of 11 harvested samples) · 11 pointer-only (licence)Dummy tokens conditioned by text query take portions of the attention weights, preventing irrelevant video clips from being represented by the text query.
-
26 May 2025 1 repository listedPodcasts have become daily companions for half a billion users.
-
2 May 2025 1 repository listedWe propose TEMPURA (Temporal Event Masked Prediction and Understanding for Reasoning in Action), a two-stage training framework that enhances video temporal understanding.
-
18 Jan 2025 1 repository listedExisting models usually first use contrastive learning methods to align video and text features, then fuse and extract multimodal information, and finally use a Transformer Decoder to decode multimodal information.
-
5 Jan 2025 1 repository listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 8 where Syntology's instrument failed) · 2 unverified (of 10 harvested samples) · 10 pointer-only (licence)In this paper, we present a novel Video Context-aware Keyword Attention module that overcomes this limitation by capturing keyword variation within the context of the entire video.
-
18 Dec 2024 1 repository listed Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 2 violated, 2 with no contract checked; 5 where Syntology's instrument failed) · 4 unverified (of 13 harvested samples) · 13 pointer-only (licence)For short-moment retrieval, FlashVTG increases mAP to 125% of previous SOTA performance.
-
12 Dec 2024 1 repository listed Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 5 where Syntology's instrument failed) · 1 unverified (of 11 harvested samples)The demand for producing short-form videos for sharing on social media platforms has experienced significant growth in recent times.
-
2 Dec 2024 1 repository listedVideo Highlight Detection and Moment Retrieval (HD/MR) are essential in video analysis.
-
27 Nov 2024 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)We construct MMDuetIT, a video-text training dataset designed to adapt VideoLLMs to video-text duet interaction format.
-
15 Nov 2024 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 7 unverified (of 11 harvested samples) · 3 pointer-only (licence)Video Large Language Models (Vid-LLMs) have made remarkable advancements in comprehending video content for QA dialogue.
-
25 Oct 2024 1 repository listed Syntology 6 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 6 harvested samples) · 2 pointer-only (licence)This paper proposes TimeSuite, a collection of new designs to adapt the existing short-form video MLLMs for long video understanding, including a simple yet efficient framework to process long video sequence, a…
-
2 Oct 2024 1 repository listedCombined with the introduced Saliency-Guided Cross Attention mechanism and a hybrid DETR architecture, our approach significantly enhances performance in both moment retrieval and highlight detection tasks.
-
6 Aug 2024 1 repository listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 1 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified (of 11 harvested samples) · 7 pointer-only (licence)Lighthouse addresses these issues by implementing a unified reproducible codebase that includes six models, three features, and five datasets.
-
21 Jul 2024 1 repository listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 2 violated, 1 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified (of 11 harvested samples) · 9 pointer-only (licence)Through a feasibility study, we demonstrate that LLM encoders effectively refine inter-concept relations in multimodal embeddings, even without being trained on textual embeddings.
-
22 May 2024 1 repository listed Syntology official (archive's flag): 15 ran · 15 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 10 where Syntology's instrument failed) · 2 unverified (of 17 harvested samples)Video Temporal Grounding (VTG) strives to accurately pinpoint event timestamps in a specific video using linguistic queries, significantly impacting downstream tasks like video browsing and editing.
-
14 Apr 2024 1 repository listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 1 violated, 2 with no contract checked; 4 where Syntology's instrument failed) · 2 unverified (of 10 harvested samples) · 10 pointer-only (licence)Video moment retrieval and highlight detection are two highly valuable tasks in video understanding, but until recently they have been jointly studied.
-
2 Apr 2024 1 repository listedVideo temporal grounding (VTG) is a fine-grained video understanding problem that aims to ground relevant clips in untrimmed videos given natural language queries.
-
2 Apr 2024 1 repository listed Syntology 4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified (of 7 harvested samples) · 4 pointer-only (licence)Multimodal and large language models (LLMs) have revolutionized the utilization of open-world knowledge, unlocking novel potentials across various tasks and applications.
-
31 Mar 2024 1 repository listedVideo temporal grounding (VTG) is a fine-grained video understanding problem that aims to ground relevant clips in untrimmed videos given natural language queries.
-
4 Jan 2024 1 repository listed Syntology official (archive's flag): 13 ran · 13 ran (of which 5 constructed an object rather than computing a result; 9 with no instrument failure: 1 honoured, 1 violated, 7 with no contract checked; 4 where Syntology's instrument failed) · 4 unverified (of 17 harvested samples) · 17 pointer-only (licence)Finally, a task cooperation module is constructed to refine the retrieval pipeline and the highlight score prediction process by utilizing the reciprocity between MR and HD.
-
28 Nov 2023 1 repository listed Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 0 violated, 4 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified (of 12 harvested samples) · 6 pointer-only (licence)Video Moment Retrieval (MR) and Highlight Detection (HD) have attracted significant attention due to the growing demand for video analysis.
-
28 Nov 2023 1 repository listedOnce trained, SHMGAN is able to generate specular-free images from a single RGB image as input; without requiring any additional external labels.
-
31 Jul 2023 1 repository listed Syntology official (archive's flag): 11 ran · 11 ran (of which 2 constructed an object rather than computing a result; 9 with no instrument failure: 2 honoured, 0 violated, 7 with no contract checked; 2 where Syntology's instrument failed) · 5 unverified (of 16 harvested samples) · 5 pointer-only (licence)Most methods in this direction develop taskspecific models that are trained with type-specific labels, such as moment retrieval (time interval) and highlight detection (worthiness curve), which limits their abilities to…
-
8 May 2023 1 repository listedVideo summarization has become an increasingly important task in the field of computer vision due to the vast amount of video content available on the internet.
-
29 Apr 2023 1 repository listedWith the increasing demand for video understanding, video moment and highlight detection (MHD) has emerged as a critical research topic.
-
26 Mar 2023 1 repository listedBased on existing efforts, this work has two observations: (1) For different annotators, labeling highlight has uncertainty, which leads to inaccurate and time-consuming annotations.
-
24 Mar 2023 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result (of 1 harvested sample) · 1 pointer-only (licence)As we observe the insignificant role of a given query in transformer architectures, our encoding module starts with cross-attention layers to explicitly inject the context of text query into video representation.
-
30 Aug 2022 1 repository listed
-
20 Jul 2022 1 repository listed
-
27 Aug 2021 1 repository listed
-
26 Aug 2021 1 repository listed
-
16 Aug 2021 1 repository listed
-
19 Jul 2020 1 repository listed
-
10 Dec 2019 1 repository listed
-
1 Sep 2018 1 repository listed
-
18 Apr 2018 1 repository listed
-
7 Aug 2017 1 repository listed
-
3 Jan 2017 1 repository listed
-
Unsupervised Transcript-assisted Video Summarization and Highlight Detection29 May 2025 0 repositories listed
-
Gameplay Highlights Generation12 May 2025 0 repositories listed
-
Automatic Detection of Intro and Credits in Video using CLIP and Multihead Attention13 Apr 2025 0 repositories listed
-
Multi-modal Fusion and Query Refinement Network for Video Moment Retrieval and Highlight Detection18 Jan 2025 0 repositories listed
-
Agent-based Video Trimming12 Dec 2024 0 repositories listed
-
Video LLMs for Temporal Reasoning in Long Videos4 Dec 2024 0 repositories listed
-
D&M: Enriching E-commerce Videos with Sound Effects by Key Moment Detection and SFX Matching23 Aug 2024 0 repositories listed
-
Unsupervised Video Highlight Detection by Learning from Audio and Visual Recurrence18 Jul 2024 0 repositories listed
-
A Multimodal Transformer for Live Streaming Highlight Prediction15 Jun 2024 0 repositories listed
-
Unsupervised Modality-Transferable Video Highlight Detection with Representation Activation Sequence Learning14 Mar 2024 0 repositories listed
-
GPTSee: Enhancing Moment Retrieval and Highlight Detection via Description-Based Similarity Features3 Mar 2024 0 repositories listed
-
29 Aug 2023 0 repositories listed
-
Show Me What I Like: Detecting User-Specific Video Highlights Using Content-Based Multi-Head Attention18 Jul 2022 0 repositories listed
-
Probing Visual-Audio Representation for Video Highlight Detection via Hard-Pairs Guided Contrastive Learning21 Jun 2022 0 repositories listed
-
19 Jun 2022 0 repositories listed
-
AntPivot: Livestream Highlight Detection via Hierarchical Attention Mechanism10 Jun 2022 0 repositories listed
-
Learning Pixel-Level Distinctions for Video Highlight Detection10 Apr 2022 0 repositories listed
-
Smart Director: An Event-Driven Directing System for Live Broadcasting11 Jan 2022 0 repositories listed
-
Contrastive Learning for Unsupervised Video Highlight Detection1 Jan 2022 0 repositories listed
-
4 Sep 2021 0 repositories listed
-
A Multi-Task Network for Joint Specular Highlight Detection and Removal19 Jun 2021 0 repositories listed
-
Audiovisual Highlight Detection in Videos11 Feb 2021 0 repositories listed
-
LIGHTS: LIGHT Specularity Dataset for specular detection in Multi-view26 Jan 2021 0 repositories listed
-
Joint Visual and Audio Learning for Video Highlight Detection1 Jan 2021 0 repositories listed
-
Temporal Cue Guided Video Highlight Detection With Low-Rank Audio-Visual Fusion1 Jan 2021 0 repositories listed
-
1 Dec 2020 0 repositories listed
-
10 Oct 2020 0 repositories listed
-
MINI-Net: Multiple Instance Ranking Network for Video Highlight Detection20 Jul 2020 0 repositories listed
-
Estimating Blink Probability for Highlight Detection in Figure Skating Videos2 Jul 2020 0 repositories listed
-
TruNet: Short Videos Generation from Long Videos via Story-Preserving Truncation14 Oct 2019 0 repositories listed
-
Unsupervised Multi-stream Highlight detection for the Game "Honor of Kings"14 Oct 2019 0 repositories listed
-
Less is More: Learning Highlight Detection from Video Duration3 Mar 2019 0 repositories listed
-
Deep Unsupervised Multi-View Detection of Video Game Stream Highlights25 Jul 2018 0 repositories listed
-
A Deep Ranking Model for Spatio-Temporal Highlight Detection from a 360 Video31 Jan 2018 0 repositories listed
-
Video Highlights Detection and Summarization with Lag-Calibration based on Concept-Emotion Mapping of Crowdsourced Time-Sync Comments1 Sep 2017 0 repositories listed
-
Highlight Detection With Pairwise Deep Ranking for First-Person Video Summarization1 Jun 2016 0 repositories listed
-
Discovering Picturesque Highlights from Egocentric Vacation Videos18 Jan 2016 0 repositories listed
Syntology lines on 18 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections