Browse State-of-the-Art › Video Understanding
Video Understanding
542 papers with code · 0 benchmarks · 56 datasets archive 2025-07-28
A crucial task of Video Understanding is to recognise and localise (in space and time) different actions or events appearing in the video.
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
No benchmark for this task in the archive.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
56 datasets whose archive record lists this task, ordered by the archive's paper count. 30 shown of 56 until expanded.
Subtasks archive 2025-07-28
7 subtasks in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 1,149 · with a code link: 542 · where Syntology ran a sample: 218 (177 with a run with no instrument failure, 41 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (218 of 1,149 tagged: 177 with a run with no instrument failure, 41 where every run was a failure of Syntology's instrument)
Page 1 of 12: papers 1 to 100 of 1,149, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
9 Feb 2021 16 repositories listed Syntology official: no sample here; runs from other or unrecorded repositories · 35 ran (of which 17 constructed an object rather than computing a result; 21 with no instrument failure: 0 honoured, 1 violated, 20 with no contract checked; 14 where Syntology's instrument failed) · 8 unverified (of 43 harvested samples) · 14 pointer-only (licence)We present a convolution-free approach to video classification built exclusively on self-attention over space and time.
-
24 Jun 2021 15 repositories listed Syntology official (archive's flag): 3 ran · 19 ran (of which 0 constructed an object rather than computing a result; 14 with no instrument failure: 1 honoured, 0 violated, 13 with no contract checked; 5 where Syntology's instrument failed) · 13 unverified (of 32 harvested samples) · 7 pointer-only (licence)The vision community is witnessing a modeling shift from CNNs to Transformers, where pure Transformer architectures have attained top accuracy on the major video recognition benchmarks.
-
20 Nov 2018 13 repositories listed Syntology official: harvested, nothing ran · 10 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 0 violated, 4 with no contract checked; 5 where Syntology's instrument failed) · 6 unverified (of 16 harvested samples) · 6 pointer-only (licence)The explosive growth in video streaming gives rise to challenges on performing video understanding at high accuracy and low computation cost.
-
21 Jun 2021 11 repositories listed Syntology official (archive's flag): 1 ran · 3 ran (of which 1 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)In this paper, we introduce a novel visual representation learning which relies on a handful of adaptively learned tokens, and which is applicable to both image and video understanding tasks.
-
23 Mar 2022 9 repositories listed Syntology official (archive's flag): 9 ran · 10 ran (of which 6 constructed an object rather than computing a result; 8 with no instrument failure: 1 honoured, 0 violated, 7 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified (of 13 harvested samples) · 12 pointer-only (licence)Pre-training video transformers on extra large-scale datasets is generally required to achieve premier performance on relatively small datasets.
-
23 May 2017 9 repositories listedThe AVA dataset densely annotates 80 atomic visual actions in 430 15-minute video clips, where actions are localized in space and time, resulting in 1.
-
5 Oct 2022 7 repositories listedThe SoccerNet 2022 challenges were the second annual video understanding challenges organized by the SoccerNet team.
-
12 May 2019 6 repositories listed Syntology community repositories only · 9 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 1 violated, 3 with no contract checked; 4 where Syntology's instrument failed) · 2 unverified (of 11 harvested samples) · 1 pointer-only (licence)The goal of this new task is simultaneous detection, segmentation and tracking of instances in videos.
-
24 Apr 2018 6 repositories listed Syntology official: harvested, nothing ran · 0 ran · 2 unverified (of 2 harvested samples)In this paper, we introduce a network architecture that takes long-term content into account and enables fast per-video processing at the same time.
-
29 Apr 2022 5 repositories listed Syntology 18 ran (of which 6 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 1 violated, 10 with no contract checked; 7 where Syntology's instrument failed) · 6 unverified (of 24 harvested samples) · 8 pointer-only (licence)Building models that can be rapidly adapted to novel tasks using only a handful of annotated examples is an open challenge for multimodal machine learning research.
-
18 Apr 2021 5 repositories listed Syntology official: harvested, nothing ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples) · 3 pointer-only (licence)In this paper, we propose a CLIP4Clip model to transfer the knowledge of the CLIP model to video-language retrieval in an end-to-end manner.
-
2 Oct 2018 5 repositories listedOur representation flow layer is a fully-differentiable layer designed to capture the `flow' of any representation channel within a convolutional neural network for action recognition.
-
21 Jun 2017 5 repositories listedIn particular, we evaluate our method on the large-scale multi-modal Youtube-8M v2 dataset and outperform all other methods in the Youtube 8M Large-Scale Video Understanding challenge.
-
14 Nov 2023 4 repositories listedLarge language models have demonstrated impressive universal capabilities across a wide range of open-ended tasks and have extended their utility to encompass multimodal conversations.
-
5 Jun 2023 4 repositories listed Syntology official (archive's flag): 7 ran · 18 ran (of which 7 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 1 violated, 7 with no contract checked; 10 where Syntology's instrument failed) · 7 unverified (of 25 harvested samples) · 9 pointer-only (licence)We present Video-LLaMA a multi-modal framework that empowers Large Language Models (LLMs) with the capability of understanding both visual and auditory content in the video.
-
17 Aug 2022 4 repositories listedWith the recent development of Deep Learning applied to Computer Vision, sport video understanding has gained a lot of attention, providing much richer information for both sport consumers and leagues.
-
4 Jun 2022 4 repositories listedWe introduce CVNets, a high-performance open-source library for training deep neural networks for visual recognition tasks, including classification, detection, and segmentation.
-
27 Sep 2021 4 repositories listedSecondly, TSM has high efficiency; it achieves a high frame rate of 74fps and 29fps for online video recognition on Jetson Nano and Galaxy Note8.
-
26 Nov 2020 4 repositories listedIn this work, we propose SoccerNet-v2, a novel large-scale corpus of manual annotations for the SoccerNet video dataset, along with open challenges to encourage more research in soccer understanding and broadcast…
-
17 Jan 2020 4 repositories listed Syntology community repositories only · 17 ran (of which 9 constructed an object rather than computing a result; 14 with no instrument failure: 0 honoured, 0 violated, 14 with no contract checked; 3 where Syntology's instrument failed) · 8 unverified (of 25 harvested samples) · 1 pointer-only (licence)In this way, a heavy temporal model is replaced by a simple interlacing operator.
-
12 Dec 2018 4 repositories listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 8 harvested samples)To understand the world, we humans constantly need to relate the present to the past, and put events in context.
-
19 Jun 2018 4 repositories listedWe then implement the most common atomic (inter)actions in the Unity3D game engine, and use our programs to "drive" an artificial agent to execute tasks in a simulated household environment.
-
30 Mar 2017 4 repositories listedWe demonstrate that using both RNNs (using LSTMs) and Temporal-ConvNets on spatiotemporal feature matrices are able to exploit spatiotemporal dynamics to improve the overall performance.
-
29 Aug 2024 3 repositories listed Syntology official (archive's flag): 12 ran · 12 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 2 honoured, 0 violated, 6 with no contract checked; 4 where Syntology's instrument failed) · 5 unverified (of 17 harvested samples)Beginning with VisualGLM and CogVLM, we are continuously exploring VLMs in pursuit of enhanced vision-language fusion, efficient higher-resolution architecture, and broader modalities and applications.
-
6 Jun 2024 3 repositories listed Syntology official (archive's flag): 1 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 3 pointer-only (licence)To address the above problems, we propose a new benchmark called MLVU (Multi-task Long Video Understanding Benchmark) for the comprehensive and in-depth evaluation of LVU.
-
14 May 2024 3 repositories listedIn this paper, we propose to squeeze the time axis of a video sequence into the channel dimension and present a lightweight video recognition network, term as SqueezeTime, for mobile video understanding.
-
28 Nov 2023 3 repositories listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 1 violated, 2 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified (of 10 harvested samples) · 2 pointer-only (licence)With the rapid development of Multi-modal Large Language Models (MLLMs), a number of diagnostic benchmarks have recently emerged to evaluate the comprehension capabilities of these models.
-
28 Nov 2023 3 repositories listed Syntology official (archive's flag): 11 ran · 11 ran (of which 2 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 0 violated, 11 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 11 harvested samples) · 3 pointer-only (licence)PVSG relates to the existing video scene graph generation (VidSGG) problem, which focuses on temporal interactions between humans and objects grounded with bounding boxes in videos.
-
17 Nov 2022 3 repositories listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 5 harvested samples) · 3 pointer-only (licence)UniFormer has successfully alleviated this issue, by unifying convolution and self-attention as a relation aggregator in the transformer format.
-
19 Oct 2022 3 repositories listed Syntology community repositories only · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 4 harvested samples)Temporal action segmentation (TAS) in videos aims at densely identifying video frames in minutes-long videos with multiple action classes.
-
9 Dec 2021 3 repositories listed
-
5 Aug 2021 3 repositories listed Syntology community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
14 Jun 2020 3 repositories listed Syntology official (archive's flag): 12 ran · 17 ran (of which 0 constructed an object rather than computing a result; 16 with no instrument failure: 1 honoured, 1 violated, 14 with no contract checked; 1 where Syntology's instrument failed) · 10 unverified (of 27 harvested samples) · 4 pointer-only (licence)
-
7 Dec 2019 3 repositories listed
-
2 Dec 2019 3 repositories listed Syntology community repositories only · 6 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 4 unverified (of 10 harvested samples) · 3 pointer-only (licence)
-
28 Jun 2025 2 repositories listed
-
20 May 2025 2 repositories listed Syntology official (archive's flag): 8 ran · 9 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 1 violated, 0 with no contract checked; 6 where Syntology's instrument failed) · 1 unverified (of 10 harvested samples)
-
20 May 2025 2 repositories listed Syntology 12 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 0 violated, 5 with no contract checked; 6 where Syntology's instrument failed) · 2 unverified (of 14 harvested samples) · 5 pointer-only (licence)
-
24 Apr 2025 2 repositories listed Syntology official: harvested, nothing ran · 7 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 6 where Syntology's instrument failed) · 5 unverified (of 12 harvested samples) · 1 pointer-only (licence)
-
2 Apr 2025 2 repositories listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
6 Aug 2024 2 repositories listed
-
31 Jul 2024 2 repositories listed
-
24 Jun 2024 2 repositories listed
-
24 Jun 2024 2 repositories listed
-
15 Apr 2024 2 repositories listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 4 harvested samples) · 4 pointer-only (licence)
-
4 Apr 2024 2 repositories listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)
-
22 Mar 2024 2 repositories listed
-
16 Mar 2024 2 repositories listed
-
15 Mar 2024 2 repositories listed
-
11 Mar 2024 2 repositories listed Syntology official (archive's flag): 3 ran · 9 ran (of which 5 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 0 violated, 5 with no contract checked; 3 where Syntology's instrument failed) · 6 unverified (of 15 harvested samples)
-
20 Feb 2024 2 repositories listed Syntology official (archive's flag): 2 ran · 3 ran (of which 1 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 1 pointer-only (licence)
-
4 Feb 2024 2 repositories listed
-
15 Dec 2023 2 repositories listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 2 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 5 unverified (of 9 harvested samples) · 9 pointer-only (licence)
-
4 Dec 2023 2 repositories listed Syntology community repositories only · 10 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 1 violated, 4 with no contract checked; 5 where Syntology's instrument failed) · 1 unverified (of 11 harvested samples) · 1 pointer-only (licence)
-
30 Nov 2023 2 repositories listed Syntology official (archive's flag): 7 ran · 11 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 3 where Syntology's instrument failed) · 6 unverified (of 17 harvested samples) · 2 pointer-only (licence)
-
27 Nov 2023 2 repositories listed
-
12 Sep 2023 2 repositories listed
-
9 Jul 2023 2 repositories listed
-
28 Jun 2023 2 repositories listed
-
19 Jun 2023 2 repositories listed
-
8 Jun 2023 2 repositories listed
-
28 Apr 2023 2 repositories listed
-
7 Dec 2022 2 repositories listed
-
6 Dec 2022 2 repositories listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
-
19 Nov 2022 2 repositories listed Syntology official (archive's flag): 3 ran · 3 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified; every one of the 3 samples that ran constructed an object rather than computing a result (of 6 harvested samples) · 6 pointer-only (licence)
-
17 Nov 2022 2 repositories listed
-
22 Sep 2022 2 repositories listed
-
20 Jul 2022 2 repositories listed Syntology official (archive's flag): 5 ran · 5 ran (of which 1 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples) · 1 pointer-only (licence)
-
5 Jul 2022 2 repositories listed
-
5 May 2022 2 repositories listed
-
24 Mar 2022 2 repositories listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples)
-
30 Jan 2022 2 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
27 Dec 2021 2 repositories listed
-
29 Nov 2021 2 repositories listed Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 5 where Syntology's instrument failed) · 2 unverified (of 11 harvested samples) · 6 pointer-only (licence)
-
12 Oct 2021 2 repositories listed Syntology official: not harvested · 0 ran · 1 unverified (of 1 harvested sample)
-
11 Oct 2021 2 repositories listed Syntology community repositories only · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples)
-
29 Sep 2021 2 repositories listed
-
26 Jul 2021 2 repositories listed Syntology official (archive's flag): 8 ran · 13 ran (of which 4 constructed an object rather than computing a result; 12 with no instrument failure: 0 honoured, 0 violated, 12 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 14 harvested samples) · 6 pointer-only (licence)
-
7 Jul 2021 2 repositories listed
-
28 Jun 2021 2 repositories listed
-
21 Jun 2021 2 repositories listed Syntology official (archive's flag): 18 ran · 18 ran (of which 0 constructed an object rather than computing a result; 15 with no instrument failure: 2 honoured, 0 violated, 13 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 19 harvested samples) · 2 pointer-only (licence)
-
18 May 2021 2 repositories listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 7 harvested samples)
-
3 Feb 2021 2 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result (of 1 harvested sample)
-
2 Feb 2021 2 repositories listed
-
7 Jan 2021 2 repositories listed
-
30 Nov 2020 2 repositories listed
-
22 Nov 2020 2 repositories listed
-
1 Oct 2020 2 repositories listed
-
8 Aug 2020 2 repositories listed
-
1 Aug 2020 2 repositories listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 1 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 10 harvested samples) · 1 pointer-only (licence)
-
20 Jul 2020 2 repositories listed Syntology 6 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 8 harvested samples) · 2 pointer-only (licence)
-
1 Jun 2020 2 repositories listed
-
1 May 2020 2 repositories listed
-
11 Mar 2020 2 repositories listed Syntology community repositories only · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
1 Nov 2019 2 repositories listed Syntology 8 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 10 harvested samples) · 10 pointer-only (licence)
-
15 Oct 2019 2 repositories listed Syntology official: harvested for another paper · 0 ran · 1 unverified (of 1 harvested sample)
-
30 May 2019 2 repositories listed
-
21 Aug 2018 2 repositories listed
-
15 Jul 2025 1 repository listed
Syntology lines on 50 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections