Video Understanding

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

56 datasets whose archive record lists this task, ordered by the archive's paper count. 30 shown of 56 until expanded.

Subtasks archive 2025-07-28

7 subtasks in the archive's task tree.

Parent tasks archive 2025-07-28

Most implemented papers archive 2025-07-28

30 shown of 542 papers with code (1,149 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Is Space-Time Attention All You Need for Video Understanding?
  9 Feb 2021 16 repositories listed Syntology ran 35 of 43 samples · 8 unverified · 14 pointer-only (licence)
  We present a convolution-free approach to video classification built exclusively on self-attention over space and time.
• Video Swin Transformer
  24 Jun 2021 15 repositories listed Syntology ran 7 of 32 samples · 25 unverified
  The vision community is witnessing a modeling shift from CNNs to Transformers, where pure Transformer architectures have attained top accuracy on the major video recognition benchmarks.
• TSM: Temporal Shift Module for Efficient Video Understanding
  20 Nov 2018 13 repositories listed Syntology ran 6 of 16 samples · 10 unverified · 4 pointer-only (licence)
  The explosive growth in video streaming gives rise to challenges on performing video understanding at high accuracy and low computation cost.
• TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?
  21 Jun 2021 11 repositories listed Syntology ran 3 of 3 samples · 0 unverified
  In this paper, we introduce a novel visual representation learning which relies on a handful of adaptively learned tokens, and which is applicable to both image and video understanding tasks.
• VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training
  23 Mar 2022 9 repositories listed Syntology ran 9 of 13 samples · 4 unverified · 12 pointer-only (licence)
  Pre-training video transformers on extra large-scale datasets is generally required to achieve premier performance on relatively small datasets.
• AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions
  23 May 2017 9 repositories listed
  The AVA dataset densely annotates 80 atomic visual actions in 430 15-minute video clips, where actions are localized in space and time, resulting in 1.
• SoccerNet 2022 Challenges Results
  5 Oct 2022 7 repositories listed
  The SoccerNet 2022 challenges were the second annual video understanding challenges organized by the SoccerNet team.
• Video Instance Segmentation
  12 May 2019 6 repositories listed Syntology ran 3 of 11 samples · 8 unverified · 1 pointer-only (licence)
  The goal of this new task is simultaneous detection, segmentation and tracking of instances in videos.
• ECO: Efficient Convolutional Network for Online Video Understanding
  24 Apr 2018 6 repositories listed Syntology ran 0 of 2 samples · 2 unverified
  In this paper, we introduce a network architecture that takes long-term content into account and enables fast per-video processing at the same time.
• Flamingo: a Visual Language Model for Few-Shot Learning
  29 Apr 2022 5 repositories listed Syntology ran 18 of 24 samples · 6 unverified · 7 pointer-only (licence)
  Building models that can be rapidly adapted to novel tasks using only a handful of annotated examples is an open challenge for multimodal machine learning research.
• CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval
  18 Apr 2021 5 repositories listed Syntology ran 3 of 4 samples · 1 unverified · 3 pointer-only (licence)
  In this paper, we propose a CLIP4Clip model to transfer the knowledge of the CLIP model to video-language retrieval in an end-to-end manner.
• Representation Flow for Action Recognition
  2 Oct 2018 5 repositories listed
  Our representation flow layer is a fully-differentiable layer designed to capture the `flow' of any representation channel within a convolutional neural network for action recognition.
• Learnable pooling with Context Gating for video classification
  21 Jun 2017 5 repositories listed
  In particular, we evaluate our method on the large-scale multi-modal Youtube-8M v2 dataset and outperform all other methods in the Youtube 8M Large-Scale Video Understanding challenge.
• Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding
  14 Nov 2023 4 repositories listed
  Large language models have demonstrated impressive universal capabilities across a wide range of open-ended tasks and have extended their utility to encompass multimodal conversations.
• Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding
  5 Jun 2023 4 repositories listed Syntology ran 18 of 25 samples · 7 unverified · 9 pointer-only (licence)
  We present Video-LLaMA a multi-modal framework that empowers Large Language Models (LLMs) with the capability of understanding both visual and auditory content in the video.
• DeepSportradar-v1: Computer Vision Dataset for Sports Understanding with High Quality Annotations
  17 Aug 2022 4 repositories listed
  With the recent development of Deep Learning applied to Computer Vision, sport video understanding has gained a lot of attention, providing much richer information for both sport consumers and leagues.
• CVNets: High Performance Library for Computer Vision
  4 Jun 2022 4 repositories listed
  We introduce CVNets, a high-performance open-source library for training deep neural networks for visual recognition tasks, including classification, detection, and segmentation.
• TSM: Temporal Shift Module for Efficient and Scalable Video Understanding on Edge Device
  27 Sep 2021 4 repositories listed
  Secondly, TSM has high efficiency; it achieves a high frame rate of 74fps and 29fps for online video recognition on Jetson Nano and Galaxy Note8.
• SoccerNet-v2: A Dataset and Benchmarks for Holistic Understanding of Broadcast Soccer Videos
  26 Nov 2020 4 repositories listed
  In this work, we propose SoccerNet-v2, a novel large-scale corpus of manual annotations for the SoccerNet video dataset, along with open challenges to encourage more research in soccer understanding and broadcast…
• Temporal Interlacing Network
  17 Jan 2020 4 repositories listed Syntology ran 12 of 25 samples · 13 unverified
  In this way, a heavy temporal model is replaced by a simple interlacing operator.
• Long-Term Feature Banks for Detailed Video Understanding
  12 Dec 2018 4 repositories listed Syntology ran 0 of 8 samples · 8 unverified
  To understand the world, we humans constantly need to relate the present to the past, and put events in context.
• VirtualHome: Simulating Household Activities via Programs
  19 Jun 2018 4 repositories listed
  We then implement the most common atomic (inter)actions in the Unity3D game engine, and use our programs to "drive" an artificial agent to execute tasks in a simulated household environment.
• TS-LSTM and Temporal-Inception: Exploiting Spatiotemporal Dynamics for Activity Recognition
  30 Mar 2017 4 repositories listed
  We demonstrate that using both RNNs (using LSTMs) and Temporal-ConvNets on spatiotemporal feature matrices are able to exploit spatiotemporal dynamics to improve the overall performance.
• CogVLM2: Visual Language Models for Image and Video Understanding
  29 Aug 2024 3 repositories listed Syntology ran 12 of 17 samples · 5 unverified
  Beginning with VisualGLM and CogVLM, we are continuously exploring VLMs in pursuit of enhanced vision-language fusion, efficient higher-resolution architecture, and broader modalities and applications.
• MLVU: Benchmarking Multi-task Long Video Understanding
  6 Jun 2024 3 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 3 pointer-only (licence)
  To address the above problems, we propose a new benchmark called MLVU (Multi-task Long Video Understanding Benchmark) for the comprehensive and in-depth evaluation of LVU.
• No Time to Waste: Squeeze Time into Channel for Mobile Video Understanding
  14 May 2024 3 repositories listed
  In this paper, we propose to squeeze the time axis of a video sequence into the channel dimension and present a lightweight video recognition network, term as \textit{SqueezeTime}, for mobile video understanding.
• Panoptic Video Scene Graph Generation
  28 Nov 2023 3 repositories listed Syntology ran 11 of 11 samples · 0 unverified · 3 pointer-only (licence)
  PVSG relates to the existing video scene graph generation (VidSGG) problem, which focuses on temporal interactions between humans and objects grounded with bounding boxes in videos.
• MVBench: A Comprehensive Multi-modal Video Understanding Benchmark
  28 Nov 2023 3 repositories listed Syntology ran 7 of 10 samples · 3 unverified
  With the rapid development of Multi-modal Large Language Models (MLLMs), a number of diagnostic benchmarks have recently emerged to evaluate the comprehension capabilities of these models.
• UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormer
  17 Nov 2022 3 repositories listed Syntology ran 3 of 5 samples · 2 unverified
  UniFormer has successfully alleviated this issue, by unifying convolution and self-attention as a relation aggregator in the transformer format.
• Temporal Action Segmentation: An Analysis of Modern Techniques
  19 Oct 2022 3 repositories listed Syntology ran 2 of 4 samples · 2 unverified
  Temporal action segmentation (TAS) in videos aims at densely identifying video frames in minutes-long videos with multiple action classes.

Syntology lines on 18 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.