Browse State-of-the-Art › Action Recognition

Action Recognition

1,058 papers with code · 56 benchmarks · 115 datasets archive 2025-07-28

Computer VisionTime Series

Action Recognition is a computer vision task that involves recognizing human actions in videos or images. The goal is to classify and categorize the actions being performed in the video or image into a predefined set of action classes.

In the video domain, it is an open question whether training an action classification network on a sufficiently large dataset, will give a similar boost in performance when applied to a different temporal task or dataset. The challenges of building video datasets has meant that most popular benchmarks for action recognition are small, having on the order of 10k videos.

Please note some benchmarks may be located in the Action Classification or Video Classification tasks, e.g. Kinetics-400.

Description from the archive archive 2025-07-28; Papers-with-Code links inside it are rewritten to this site.

Benchmarks archive 2025-07-28

56 leaderboard tables shown for this task, 56 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted. 10 shown of 56 until expanded.

Syntology 2 newer papers were judged to report on this task's benchmarks so far (Claude Sonnet 4.5, reading their tables: a model's judgement, not a result; 0 placed): NTU RGB+D (1 judged, 0 placed), UTD-MHAD (1 judged, 0 placed). Drawn from 8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read).

DatasetBest model (first row in archive order)PaperCodeSyntologyCompare
Something-Something V2 (123 rows) MVD (Kinetics400 pretrain, ViT-H, 16 frame) Masked Video Distillation: Rethinking Masked Feature Modeling for... code — Compare
UCF101 (91 rows) FTP-UniFormerV2-L/14 Enhancing Video Transformers for Action Understanding with... — — Compare
HMDB-51 (77 rows) VideoMAE V2-g VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking code Syntology official (archive's flag): 2 ran · 2 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 4 unverified; every one of the 2 samples that ran constructed an object rather than computing a result Compare
Something-Something V1 (74 rows) InternVideo InternVideo: General Video Foundation Models via Generative and... code Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified Compare
AVA v2.2 (38 rows) LART (Hiera-H, K700 PT+FT) On the Benefits of 3D Pose and Tracking for Human Action Recognition code — Compare
EPIC-KITCHENS-100 (32 rows) LLaVAction LLaVAction: evaluating and training multi-modal large language... code — Compare
NTU RGB+D (28 rows) DSCNet (RGB + Pose) A Dense-Sparse Complementary Network for Human Action Recognition... code — Compare
NTU RGB+D 120 (21 rows) DSCNet (RGB + Pose) A Dense-Sparse Complementary Network for Human Action Recognition... code — Compare
Diving-48 (18 rows) LVMAE Extending Video Masked Autoencoders to 128 frames — — Compare
ActivityNet (16 rows) Text4Vis (w/ ViT-L) Revisiting Classifier: Transferring Vision-Language Models for... code — Compare
AVA v2.1 (15 rows) STAR/L End-to-End Spatio-Temporal Action Localisation with Video Transformers — — Compare
H2O (2 Hands and Objects) (11 rows) HandFormer-B/21x8 On the Utility of 3D Hand Poses for Action Recognition code Syntology official (archive's flag): 8 ran · 8 ran (of which 5 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 1 violated, 5 with no contract checked; 2 where Syntology's instrument failed) · 5 unverified Compare
THUMOS’14 (10 rows) BMN BMN: Boundary-Matching Network for Temporal Action Proposal Generation code Syntology community repositories only · 11 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 2 honoured, 0 violated, 8 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified Compare
Sports-1M (9 rows) ip-CSN-152 (RGB) Video Classification with Channel-Separated Convolutional Networks code Syntology official (archive's flag): 1 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified Compare
HACS (8 rows) InternVideo2-6B InternVideo2: Scaling Foundation Models for Multimodal Video Understanding code — Compare
Charades-Ego (6 rows) LaViLa (Finetuned, TimeSformer-L) Learning Video Representations from Large Language Models code Syntology official (archive's flag): 13 ran · 13 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 2 violated, 7 with no contract checked; 4 where Syntology's instrument failed) · 7 unverified Compare
Volleyball (4 rows) PoseC3D (Pose Only) Revisiting Skeleton-based Action Recognition code — Compare
Animal Kingdom (4 rows) ARTEMIS: animal recognition through enhanced multimodal integration system — — — Compare
BAR (4 rows) DebiAN Discover and Mitigate Unknown Biases with Debiasing Alternate Networks code Syntology official (archive's flag): 2 ran · 2 ran (of which 1 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 7 unverified Compare
HAA500 (4 rows) TSN HAA500: Human-Centric Atomic Action Dataset with Curated Videos — — Compare
LoTE-Animal (4 rows) SlowOnly r50 LoTE-Animal: A Long Time-span Dataset for Endangered Animal... — — Compare
UAV-Human (4 rows) PMI Sampler PMI Sampler: Patch Similarity Guided Frame Selection for Aerial... code — Compare
Jester (Gesture Recognition) (3 rows) DirecFormer DirecFormer: A Directed Attention in Transformer Approach to... code Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified Compare
RareAct (3 rows) 🦩 Flamingo Flamingo: a Visual Language Model for Few-Shot Learning code Syntology 18 ran (of which 6 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 1 violated, 10 with no contract checked; 7 where Syntology's instrument failed) · 6 unverified Compare
Real Life Violence Situations Dataset (3 rows) DeVTr Data Efficient Video Transformer for Violence Detection code — Compare
ICVL-4 (2 rows) OHA-GCN (Two stream; HP + OHP-hands + informative samples) Skeleton-based Action Recognition of People Handling Objects — — Compare
IRD (2 rows) OHA-GCN (Two stream; HP + OHP-hands + informative samples) Skeleton-based Action Recognition of People Handling Objects — — Compare
miniSports (2 rows) IF+MD+RGB-R (ResNet-18) SCSampler: Sampling Salient Clips from Video for Efficient Action... — — Compare
UCF-101 (2 rows) DMC-Net (ResNet-18) DMC-Net: Generating Discriminative Motion Cues for Fast Compressed... — — Compare
Drone-Action (2 rows) FAR FAR: Fourier Aerial Video Recognition code — Compare
Mimetics (2 rows) JMRN Pose And Joint-Aware Action Recognition code — Compare
Okutama-Action (2 rows) PLAR with bbox (Ours) SCP: Soft Conditional Prompt Learning for Aerial Video Action Recognition — — Compare
Penn Action (2 rows) 3DA (RGB + Pose) Cross-Modal Learning with 3D Deformable Attention for Action Recognition — — Compare
SL-Animals (2 rows) SEW-Resnet18 (3sets) EventRPG: Event Data Augmentation with Relevance Propagation Guidance code Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 11 unverified Compare
ActionNet-VE (1 row) Baseline Extensible Hierarchical Method of Detecting Interactive Actions... — — Compare
Charades (1 row) MSQNet Actor-agnostic Multi-label Action Recognition with Multi-modal Query code — Compare
EgoGesture (1 row) TSM+W3 Knowing What, Where and When to Look: Efficient Video Action... — — Compare
EPIC-KITCHENS-55 (1 row) TSM+W3 - full res Knowing What, Where and When to Look: Efficient Video Action... — — Compare
HMDB51 (1 row) MSQNet Actor-agnostic Multi-label Action Recognition with Multi-modal Query code — Compare
UTD-MHAD (1 row) Action Machine (RGB only) Action Machine: Rethinking Action Recognition in Trimmed Videos — — Compare
VIRAT Ground 2.0 (1 row) DHCM A Hierarchical Context Model for Event Recognition in Surveillance Video — — Compare
DVS128 Gesture (1 row) SEW-Resnet18 EventRPG: Event Data Augmentation with Relevance Propagation Guidance code Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 11 unverified Compare
Hockey (1 row) MSQNet Actor-agnostic Multi-label Action Recognition with Multi-modal Query code — Compare
IndustReal (1 row) MViT-V2 IndustReal: A Dataset for Procedure Step Recognition Handling... code Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified Compare
KTH (1 row) CNN-GRU Temporal Relations of Informative Frames in Action Recognition code — Compare
MECCANO (1 row) SlowFast The MECCANO Dataset: Understanding Human-Object Interactions from... code — Compare
MTL-AQA (1 row) C3D-AVG What and How Well You Performed? A Multitask Learning Approach to... code — Compare
N-UCLA (1 row) DVANet DVANet: Disentangling View and Action Features for Multi-View... code — Compare
NEC Drone (1 row) FAR FAR: Fourier Aerial Video Recognition code — Compare
RoCoG-v2 (1 row) AZTR (Ours) AZTR: Aerial Video Action Recognition with Auto Zoom and Temporal Reasoning — — Compare
Skeleton-Mimetics (1 row) Structured Keypoint Pooling Unified Keypoint-based Action Recognition Framework via Structured... — — Compare
THUMOS14 (1 row) MSQNet Actor-agnostic Multi-label Action Recognition with Multi-modal Query code — Compare
UAV Human (1 row) FAR FAR: Fourier Aerial Video Recognition code — Compare
UCF 101 (1 row) R2+1D-BERT Late Temporal Modeling in 3D CNN Architectures with BERT for... code — Compare
UCFSports (1 row) CNN-LSTM Temporal Relations of Informative Frames in Action Recognition code — Compare
Win-Fail Action Understanding (1 row) 2DCNN+TRN Win-Fail Action Recognition code — Compare

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

115 datasets whose archive record lists this task, ordered by the archive's paper count. 30 shown of 115 until expanded.

Subtasks archive 2025-07-28

15 subtasks in the archive's task tree.

Parent tasks archive 2025-07-28

Papers archive 2025-07-28

archive papers tagged: 2,759 · with a code link: 1,058 · where Syntology ran a sample: 275 (228 with a run with no instrument failure, 47 where every run was a failure of Syntology's instrument) Syntology

Show: all tagged papersonly where code ran (275 of 2,759 tagged: 228 with a run with no instrument failure, 47 where every run was a failure of Syntology's instrument)

Page 1 of 28: papers 1 to 100 of 2,759, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.

Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.

Syntology lines on 56 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections