</> Code with Papers by Syntology

Datasets › Kinetics

Kinetics (Kinetics Human Action Video Dataset)

Introduced by Will Kay et al. in The Kinetics Human Action Video Dataset19 May 2017 archive 2025-07-28

The Kinetics dataset is a large-scale, high-quality dataset for human action recognition in videos. The dataset consists of around 500,000 video clips covering 600 human action classes with at least 600 video clips for each action class. Each video clip lasts around 10 seconds and is labeled with a single action class. The videos are collected from YouTube.

Source: Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey

HomepageBenchmarksPapers JSON

Benchmarks archive 2025-07-28

All 18 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.

First row (archive order)PaperCode
Action Classification Kinetics-400 OmniVec2 Acc@1 93.6 OmniVec2 - A Novel Transformer based Network for Large... — 207 Compare
Skeleton Based Action Recognition Kinetics-Skeleton dataset Structured Keypoint Pooling (PPNv2 skeletons+objects) Accuracy 52.3 Unified Keypoint-based Action Recognition Framework via... — 42 Compare
Zero-Shot Action Recognition Kinetics TC-CLIP Top-1 Accuracy 78.1 Leveraging Temporal Contextualization for Video Action... naver-ai/tc-clip +1 20 Compare
Video Prediction Kinetics-600 12 frames, 64x64 SiD2 FVD 2.3 Simpler Diffusion (SiD2): 1.5 FID on ImageNet512 with... — 16 Compare
Action Classification Kinetics-Sounds CA2ST(B/16) Top 1 Accuracy 93.3 CA^2ST: Cross-Attention in Audio, Space, and Time for... — 4 Compare
Self-Supervised Action Recognition Kinetics-400 XKD (ViT-B/112/16) Top-1 accuracy % 77.6 XKD: Cross-modal Knowledge Distillation with Domain... pritamqu/XKD 4 Compare
Video Generation Kinetics-600 12 frames, 64x64 W.A.L.T-L FVD 3.3±0.0 Photorealistic Video Generation with Diffusion Models — 4 Compare
Action Recognition In Videos Kinetics-400 Florence Top-1 Accuracy 86.5 Florence: A New Foundation Model for Computer Vision microsoft/unicl +1 3 Compare
Visual Tracking Kinetics TAPIR (Panning MOVi-E) Average Jaccard 57.2 TAPIR: Tracking Any Point with per-frame Initialization... deepmind/tapnet +2 2 Compare
Action Classification MiniKinetics MARS+RGB+Flow (16 frames) Top-1 Accuracy 73.5 MARS: Motion-Augmented RGB Stream for Action Recognition craston/MARS 1 Compare
Boundary Detection Kinetics-400 CASTANET+ Ensemble Pairwise F1 0.814 Generic Event Boundary Detection Challenge at CVPR 2021... DexiangHong/Cascade-PC 1 Compare
Event Segmentation Kinetics-400 CASTANET+ Ensemble F1 83.3 Generic Event Boundary Detection Challenge at CVPR 2021... DexiangHong/Cascade-PC 1 Compare
Generic Event Boundary Detection Kinetics-GEBD FlowGEBD F1 @ RelDis. 0.05 0.713 What's in the Flow? Exploiting Temporal Motion Cues for... — 1 Compare
Skeleton Based Action Recognition Kinetics-400 STGAT Actions Top-1 (S1) 39.2 Spatial Temporal Graph Attention Network for... hulianyuyy/STGAT 1 Compare
Text-to-Video Generation Kinetics NUWA (128×128) Accuracy 77.9 NÜWA: Visual Synthesis Pre-training for Neural visUal... lucidrains/nuwa-pytorch 1 Compare
Video Classification Kinetics Multigrid Top-1 77.6 A Multigrid Method for Efficiently Training Video Models facebookresearch/SlowFast +2 1 Compare
Video Generation Kinetics-600 12 frames, 128x128 DVD-GAN FID 2.16 Adversarial Video Generation on Complex Datasets Harrypotterrrr/DVD-GAN 1 Compare
Video Generation Kinetics-600 48 frames, 64x64 DVD-GAN FID 12.92 Adversarial Video Generation on Complex Datasets Harrypotterrrr/DVD-GAN 1 Compare

Papers archive 2025-07-28

30 shown of 171 papers with a leaderboard row on this dataset's benchmarks, newest first. The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset; the archive's count for this dataset is 1,341. The Syntology column is from Syntology's graph (read 2026-09-24), stated per sample; it is not part of any archive number.

DateSamples run Syntology
CA^2ST: Cross-Attention in Audio, Space, and Time for Holistic Video Recognition 0 2 30 Mar 2025 not harvested
Make Your Training Flexible: Towards Deployment-Efficient Video Models 1 2 18 Mar 2025 ran 7 of 17 samples (10 unverified)
DejaVid: Encoder-Agnostic Learned Temporal Matching for Video Classification 1 1 1 Jan 2025 not harvested
Revealing Key Details to See Differences: A Novel Prototypical Perspective for Skeleton-based Action Recognition 1 1 28 Nov 2024 not harvested
LoCATe-GAT: Modeling Multi-Scale Local Context and Action Relationships for Zero-Shot Action Recognition 1 1 27 Nov 2024 not harvested
AM Flow: Adapters for Temporal Processing in Action Recognition 0 1 4 Nov 2024 not harvested
LARP: Tokenizing Videos with a Learned Autoregressive Generative Prior 1 1 28 Oct 2024 ran 4 of 13 samples (9 unverified)
Simpler Diffusion (SiD2): 1.5 FID on ImageNet512 with pixel-space diffusion 0 1 25 Oct 2024 not harvested
OmniTokenizer: A Joint Image-Video Tokenizer for Visual Generation 1 1 13 Jun 2024 ran 5 of 5 samples (0 unverified; 1 pointer-only for licence)
Leveraging Temporal Contextualization for Video Action Recognition 2 1 15 Apr 2024 ran 2 of 4 samples (2 unverified; 4 pointer-only for licence)
Learning Correlation Structures for Vision Transformers 0 1 5 Apr 2024 not harvested
Enhancing Video Transformers for Action Understanding with VLM-aided Training 0 1 24 Mar 2024 not harvested
InternVideo2: Scaling Foundation Models for Multimodal Video Understanding 2 2 22 Mar 2024 not harvested
VideoMamba: State Space Model for Efficient Video Understanding 2 1 11 Mar 2024 ran 9 of 15 samples (6 unverified)
What's in the Flow? Exploiting Temporal Motion Cues for Unsupervised Generic Event Boundary Detection 0 1 15 Feb 2024 not harvested
OmniVec2 - A Novel Transformer based Network for Large Scale Multimodal and Multitask Learning 0 1 1 Jan 2024 not harvested
Photorealistic Video Generation with Diffusion Models 0 2 11 Dec 2023 not harvested
OST: Refining Text Knowledge with Optimal Spatio-Temporal Descriptor for General Video Recognition 1 1 30 Nov 2023 ran 1 of 2 samples (1 unverified)
CAST: Cross-Attention in Space and Time for Video Action Recognition 1 1 30 Nov 2023 ran 9 of 17 samples (8 unverified; 17 pointer-only for licence)
Side4Video: Spatial-Temporal Side Network for Memory-Efficient Image-to-Video Transfer Learning 2 1 27 Nov 2023 not harvested
Mirasol3B: A Multimodal Autoregressive model for time-aligned and contextual modalities 0 1 9 Nov 2023 not harvested
OmniVec: Learning robust representations with cross modal sharing 0 1 7 Nov 2023 not harvested
Asymmetric Masked Distillation for Pre-Training Small Foundation Models 0 2 6 Nov 2023 not harvested
Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation 3 2 9 Oct 2023 ran 12 of 20 samples (8 unverified)
LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment 6 1 3 Oct 2023 ran 7 of 14 samples (7 unverified)
ZeroI2V: Zero-Cost Adaptation of Pre-trained Transformers from Image to Video 2 1 2 Oct 2023 ran 1 of 3 samples (2 unverified)
Orthogonal Temporal Interpolation for Zero-Shot Video Recognition 1 1 14 Aug 2023 ran 3 of 3 samples (0 unverified; 3 pointer-only for licence)
Temporally-Adaptive Models for Efficient Video Understanding 1 2 10 Aug 2023 ran 2 of 2 samples (0 unverified)
What Can Simple Arithmetic Operations Do for Temporal Modeling? 2 1 18 Jul 2023 ran 3 of 6 samples (3 unverified)
TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement 3 2 14 Jun 2023 ran 0 of 1 samples (1 unverified)

The full list of 171 is in the JSON twin.

Dataset loaders archive 2025-07-28

pytorch.org/vision/stable/generated/torchvision.datasets.Kinetics.htmlpytorch/visionpytorch
open-mmlab/mmaction2open-mmlab/mmaction2pytorch

2 loaders as listed in the archive; links are outbound and not re-checked here.

Tasks archive 2025-07-28

Action RecognitionTemporal Action LocalizationImage ClusteringSkeleton Based Action RecognitionVideo RetrievalAction ClassificationAction Recognition In VideosLong-tail LearningVideo PredictionVideo UnderstandingVisual TrackingVideo CaptioningVideo GenerationVideo ClassificationBoundary DetectionZero-Shot Action RecognitionSelf-Supervised Action RecognitionText-to-Video GenerationText to Video RetrievalFew Shot Action RecognitionSemantic Object Interaction ClassificationEvent SegmentationVideo GroundingSelf-Supervised Action Recognition LinearGeneric Event Boundary DetectionSpatio-Temporal Action LocalizationBoundary CaptioningBoundary GroundingVideo Recognitionimbalanced classification

License archive 2025-07-28

CC BY 4.0

Modalities archive 2025-07-28

Videos

Languages archive 2025-07-28

No language tagged.

Variants archive 2025-07-28

  • Kinetics-GEBD
  • MiniKinetics
  • Kinetics-ZSAR
  • Kinetics-Sounds
  • Kinetics-Sound
  • Kinetics-GEB+
  • Kinetics-700
  • Kinetics-600 12 frames, 64x64
  • Kinetics-600 12 frames, 128x128
  • Kinetics-600
  • Kinetics 400
  • Kinetics-100
  • Imbalanced-MiniKinetics200
  • Kinetics-Skeleton dataset
  • AVA-Kinetics
  • Kinetics-600 48 frames, 64x64
  • Kinetics-400
  • Kinetics

18 variant names, as the archive lists them.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections