Papers › Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset

Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset

22 May 2017CVPR 2017 7arXiv:1705.07750archive 2025-07-28

Joao Carreira, Andrew Zisserman

The paucity of videos in current action classification datasets (UCF-101 and HMDB-51) has made it difficult to identify good video architectures, as most methods obtain similar performance on existing small-scale benchmarks. This paper re-evaluates state-of-the-art architectures in light of the new Kinetics Human Action Video dataset. Kinetics has two orders of magnitude more data, with 400 human action classes and over 400 clips per class, and is collected from realistic, challenging YouTube videos. We provide an analysis on how current architectures fare on the task of action classification on this dataset and how much performance improves on the smaller benchmark datasets after pre-training on Kinetics. We also introduce a new Two-Stream Inflated 3D ConvNet (I3D) that is based on 2D ConvNet inflation: filters and pooling kernels of very deep image classification ConvNets are expanded into 3D, making it possible to learn seamless spatio-temporal feature extractors from video while leveraging successful ImageNet architecture designs and even their parameters. We show that, after pre-training on Kinetics, I3D models considerably improve upon the state-of-the-art in action classification, reaching 80.9% on HMDB-51 and 98.0% on UCF-101.

PaperPDFConference PDFCodeCode Syntology ran

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, Syntology's MCP tool lists every function and class Syntology harvested from this paper and whether it ran (how to connect): get_harvested_code_for_paper(arxiv_id="1705.07750")

Code

Syntology Ran 16 of 28 code samples harvested from 8 repositories linked to this paper; 12 have no recorded run. Of those that ran: 3 ran · our draft was wrong; 3 ran · fixture could not drive it; 10 ran with no contract checked.

By repository: community (archive-listed): 24 samples from 8 repositories, 12 ran; 4 identical to code first harvested elsewhere. The run record, sample by sample. “Ran” means executed on a synthesized input, not that the code is correct or reproduces the paper.

34 repositories listed; official and paper-mentioned ones first.

2023-MindSpore-1/ms-code-24 mentioned on GitHubmindspore report
Alexyuda/action_recognition mentioned on GitHubpytorch report
CMU-CREATE-Lab/deep-smoke-machine mentioned on GitHubpytorch report
FrederikSchorr/sign-language mentioned on GitHubtf report
JeffCHEN2017/WSSTG mentioned on GitHubpytorchNOASSERTION report
KingGugu/I3D mentioned on GitHubmindspore report
LukasHedegaard/co3d mentioned on GitHubpytorchApache-2.0 report
OanaIgnat/i3d_keras mentioned on GitHubtf report
PPPrior/i3d-pytorch mentioned on GitHubpytorch report
StanfordVL/RubiksNet mentioned on GitHubpytorch report
aim3-ruc/youmakeup_challenge2022 mentioned on GitHubpytorch report
daniansan/i3d_mindspore mentioned on GitHubmindspore report
deepmind/kinetics-i3d mentioned on GitHubtf report
dlpbc/keras-kinetics-i3d mentioned on GitHubtf report
google-deepmind/kinetics-i3d mentioned on GitHubtf report
hassony2/kinetics_i3d_pytorch mentioned on GitHubpytorch report
helloxy96/CS5242_Project2020 mentioned on GitHubpytorch report
mHealthBuet/SegCodeNet mentioned on GitHubpytorch report
piergiaj/pytorch-i3d mentioned on GitHubpytorch report
prinshul/GWSDR mentioned on GitHubtf report
sebastiantiesmeyer/deeplabchop3d mentioned on GitHubpytorch report
vijayvee/behavior-recognition mentioned on GitHubtf report
vijayvee/behavior_recognition mentioned on GitHubtf report
yaohungt/GSTEG_CVPR_2019 mentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

28 samples harvested; 16 ran; 0 honoured the contract we drafted; 12 have no recorded run. Read from Syntology's graph 2026-09-24; that is when this build read the record, not when the samples ran.

3ran · our draft was wrong
3ran · fixture could not drive it
10ran
12unverified

Licence: 7 of the 28 samples are pointer only, meaning Syntology does not serve that copy's text. This page shows no code text for any sample; each one links to its file in the repository.

Harvested from 8 repositories linked to this paper, official or community; each sample names its own and says which. Some samples are identical code Syntology first harvested from another repository; for those, this paper's copy is not located and its licence is not recorded. “Ran” means the sample executed on a synthesized input. It does not mean the output is correct, and nothing here reproduces the paper's results. “Honoured” and “violated” refer to a contract Syntology drafted from the code itself; “our draft was wrong” and “fixture could not drive it” are failures of Syntology's instrument, not of the code.

Each sample ends with its code_sha256, Syntology's identity for that exact code. An agent fetches the stored sample with Syntology's MCP tool get_code(code_sha256="…") (how to connect); click an identity to copy that call.

Repository labels, per sample. official repository: The archive marks this repository official for the paper. named in the paper: The archive records that the paper mentions this repository; it is not marked official. community (archive-listed): In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper. found in paper text by Syntology: Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted. community: Not in the archive's code links for this paper; a community repository Syntology harvested. Samples from a repository marked official are listed first. Licence labels name the repository's licence as recorded at harvest. “Pointer only” means Syntology does not serve that copy's text, for one of four reasons: no licence file was found; the licence was not identified; the licence is recorded as permissive but that copy's record is not marked cleared; or the licence is outside the permissive list Syntology serves text under (MIT, Apache-2.0, BSD and similar). Some licences outside that list permit redistribution, such as WTFPL, and GPL-3.0 under its conditions; they are simply not on the list. Hover a licence label for the reason. File links open the file on GitHub at the default branch, which may have changed since the harvest.

I3D PPPrior/i3d-pytorch/models/i3d.py community (archive-listed) ran no licence file found · pointer only · d52425813034770e · report
I3dRgbUcf101 Alexyuda/action_recognition/Models.py community (archive-listed) ran no licence file found · pointer only · afb1546ff0edeb60 · report
InceptionModule CMU-CREATE-Lab/deep-smoke-machine/back-end/www/model/pytorch_i3d.py community (archive-listed) ran · metamorphic tier: deterministic BSD-3-Clause (permissive) · 86ab28ecf5cc710e · report
MaxPool3dSamePadding CMU-CREATE-Lab/deep-smoke-machine/back-end/www/model/pytorch_i3d.py community (archive-listed) ran · metamorphic tier: deterministic BSD-3-Clause (permissive) · 0cca1672c970b785 · report
MaxPool3dTFPadding ivanwilliammd/I3DR-Net-Transfer-Learning/models/I3D_model/i3dpt.py community (archive-listed) ran · metamorphic tier: deterministic fingerprinted Apache-2.0 (permissive) · 162d0ddf9cfd939e · report
MaxPool3dTFPadding hassony2/kinetics_i3d_pytorch/src/i3dpt.py community (archive-listed) ran · metamorphic tier: invariant fingerprinted MIT (permissive) · a0d388f676f51bcf · report
ResNet sebastiantiesmeyer/deeplabchop3d/models/resnet.py community (archive-listed) ran · metamorphic tier: deterministic MIT (permissive) · 8f8c0c36c90310ff · report
Unit3D CMU-CREATE-Lab/deep-smoke-machine/back-end/www/model/pytorch_i3d.py community (archive-listed) ran · metamorphic tier: deterministic BSD-3-Clause (permissive) · 8f662573b6fac87b · report
Unit3Dpy ivanwilliammd/I3DR-Net-Transfer-Learning/models/I3D_model/i3dpt.py community (archive-listed) ran · metamorphic tier: deterministic Apache-2.0 (permissive) · 7db92abddea0028b · report
Unit3Dpy hassony2/kinetics_i3d_pytorch/src/i3dpt.py community (archive-listed) ran MIT (permissive) · 7d55ae5d78e79327 · report
_get_padding ivanwilliammd/I3DR-Net-Transfer-Learning/models/I3D_model/i3dpt.py community (archive-listed) ran · our draft was wrong Apache-2.0 (permissive) · 42f2e711eec1d11f · report
get_padding_shape hassony2/kinetics_i3d_pytorch/src/i3dpt.py community (archive-listed) ran · fixture could not drive it MIT (permissive) · c84c49b2d9ed0ea8 · report
I3D ivanwilliammd/I3DR-Net-Transfer-Learning/models/I3D_model/i3dpt.py community (archive-listed) unverified Apache-2.0 (permissive) · 4165130355776ce6 · report
I3D hassony2/kinetics_i3d_pytorch/src/i3dpt.py community (archive-listed) unverified MIT (permissive) · c124939daa593331 · report
InceptionI3d CMU-CREATE-Lab/deep-smoke-machine/back-end/www/model/pytorch_i3d.py community (archive-listed) unverified BSD-3-Clause (permissive) · ef9a979720d596db · report
InceptionI3d yaohungt/GSTEG_CVPR_2019/models/i3d.py community (archive-listed) unverified no licence file found · pointer only · 2e635f36d98dce68 · report
InceptionI3d piergiaj/pytorch-i3d/pytorch_i3d.py community (archive-listed) unverified Apache-2.0 (permissive) · 434390ea358bf40f · report
Mixed ivanwilliammd/I3DR-Net-Transfer-Learning/models/I3D_model/i3dpt.py community (archive-listed) unverified Apache-2.0 (permissive) · 1d5fe61bcc4a9357 · report
Mixed hassony2/kinetics_i3d_pytorch/src/i3dpt.py community (archive-listed) unverified MIT (permissive) · 1f304ea60f4c8fe0 · report
get_bn_params ivanwilliammd/I3DR-Net-Transfer-Learning/models/I3D_model/i3dpt.py community (archive-listed) unverified Apache-2.0 (permissive) · 040b15375c9a9f1d · report
load_conv3d ivanwilliammd/I3DR-Net-Transfer-Learning/models/I3D_model/i3dpt.py community (archive-listed) unverified Apache-2.0 (permissive) · c8b696aa9e2edd20 · report
load_conv3d hassony2/kinetics_i3d_pytorch/src/i3dpt.py community (archive-listed) unverified MIT (permissive) · 5864cfb44b943815 · report
load_mixed ivanwilliammd/I3DR-Net-Transfer-Learning/models/I3D_model/i3dpt.py community (archive-listed) unverified Apache-2.0 (permissive) · 06d60d111918490e · report
load_mixed hassony2/kinetics_i3d_pytorch/src/i3dpt.py community (archive-listed) unverified MIT (permissive) · 42cfae5f88305c31 · report
downsample_basic_block identical code first harvested elsewhere ran · our draft was wrong licence of this copy not recorded · 43c0211ff8dbca5f · report
get_conv_params identical code first harvested elsewhere ran · our draft was wrong licence of this copy not recorded · 0b19df9669841d40 · report
get_padding_shape identical code first harvested elsewhere ran · fixture could not drive it fingerprinted licence of this copy not recorded · 45cc344e60f640a1 · report
simplify_padding identical code first harvested elsewhere ran · fixture could not drive it fingerprinted licence of this copy not recorded · 81ec17604dc709cc · report

Tasks

Action ClassificationAction RecognitionGeneral ClassificationSkeleton Based Action RecognitionVideo Object Trackingimage-classification

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Action Classification Charades I3D MAP 32.9 #39 of 49 Archive leaderboard report
Action Classification Kinetics-400 I3D Acc@1 71.1 #179 of 207 Archive leaderboard report
Action Classification Kinetics-400 I3D Acc@5 89.3 #179 of 207 Archive leaderboard report
Action Classification MiT I3D Top 1 Accuracy 29.51% #23 of 29 Archive leaderboard report
Action Classification MiT I3D Top 5 Accuracy 56.06% #23 of 29 Archive leaderboard report
Action Classification Toyota Smarthome dataset I3D CS 53.4 #10 of 13 Archive leaderboard report
Action Classification Toyota Smarthome dataset I3D CV1 34.9 #10 of 13 Archive leaderboard report
Action Classification Toyota Smarthome dataset I3D CV2 45.1 #10 of 13 Archive leaderboard report
Action Recognition HMDB-51 Two-stream I3D Average accuracy of 3 splits 80.9 #20 of 77 Archive leaderboard report
Action Recognition HMDB-51 Two-Stream I3D (Imagenet+Kinetics pre-training) Average accuracy of 3 splits 80.7 #21 of 77 Archive leaderboard report
Action Recognition HMDB-51 Flow-I3D (Kinetics pre-training) Average accuracy of 3 splits 77.3 #33 of 77 Archive leaderboard report
Action Recognition HMDB-51 Flow-I3D (Imagenet+Kinetics pre-training) Average accuracy of 3 splits 77.1 #34 of 77 Archive leaderboard report
Action Recognition HMDB-51 RGB-I3D (Imagenet+Kinetics pre-training) Average accuracy of 3 splits 74.8 #40 of 77 Archive leaderboard report
Action Recognition HMDB-51 RGB-I3D (Kinetics pre-training) Average accuracy of 3 splits 74.3 #44 of 77 Archive leaderboard report
Action Recognition UCF101 Two-Stream I3D (Imagenet+Kinetics pre-training) 3-fold Accuracy 98.0 #12 of 91 Archive leaderboard report
Action Recognition UCF101 Two-Stream I3D (Kinetics pre-training) 3-fold Accuracy 97.8 #15 of 91 Archive leaderboard report
Action Recognition UCF101 Flow-I3D (Imagenet+Kinetics pre-training) 3-fold Accuracy 96.7 #33 of 91 Archive leaderboard report
Action Recognition UCF101 Flow-I3D (Kinetics pre-training) 3-fold Accuracy 96.5 #36 of 91 Archive leaderboard report
Action Recognition UCF101 RGB-I3D (Imagenet+Kinetics pre-training) 3-fold Accuracy 95.6 #43 of 91 Archive leaderboard report
Action Recognition UCF101 RGB-I3D (Kinetics pre-training) 3-fold Accuracy 95.1 #48 of 91 Archive leaderboard report
Action Recognition UCF101 Two-stream I3D 3-fold Accuracy 93.4 #60 of 91 Archive leaderboard report
Hand Gesture Recognition EgoGesture I3D Accuracy 92.78 #4 of 4 Archive leaderboard report
Hand Gesture Recognition VIVA Hand Gestures Dataset I3D Accuracy 83.1 #2 of 3 Archive leaderboard report
Skeleton Based Action Recognition J-HMDB I3D Accuracy (RGB+pose) 84.1 #4 of 13 Archive leaderboard report
Video Object Tracking CATER I3D-50 + LSTM L1 1.2 #7 of 7 Archive leaderboard report
Video Object Tracking CATER I3D-50 + LSTM Top 1 Accuracy 60.2 #7 of 7 Archive leaderboard report
Video Object Tracking CATER I3D-50 + LSTM Top 5 Accuracy 81.8 #7 of 7 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections