Browse State-of-the-Art › Action Understanding
Action Understanding
35 papers with code · 1 benchmark · 4 datasets archive 2025-07-28
Benchmarks archive 2025-07-28
1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| Win-Fail Action Understanding (1 row) | 2DCNN+TRN | Win-Fail Action Recognition | code | — | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
4 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 88 · with a code link: 35 · where Syntology ran a sample: 11 (8 with a run with no instrument failure, 3 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (11 of 88 tagged: 8 with a run with no instrument failure, 3 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 88 of 88, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
8 Aug 2023 2 repositories listedMoreover, combining these two paradigms in a naive manner leaves the synergy between them untapped and can lead to interference in training.
-
LLaVA-Pose: Enhancing Human Pose and Action Understanding via Keypoint-Integrated Instruction Tuning26 Jun 2025 1 repository listedWe fine-tune the LLaVA-1.
-
11 Apr 2025 1 repository listed Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 3 honoured, 0 violated, 4 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified (of 12 harvested samples) · 12 pointer-only (licence)Analyzing Fast, Frequent, and Fine-grained (F³) events presents a significant challenge in video analytics and multi-modal LLMs.
-
24 Mar 2025 1 repository listedThe recent development of multi-modal large language models (MLLMs) is a promising candidate for a wide range of action understanding tasks.
-
2 Jan 2025 1 repository listedExperiments show that SeFAR achieves state-of-the-art performance on two FAR datasets, FineGym and FineDiving, across various data scopes.
-
31 Oct 2024 1 repository listedRegarding action relationships, the Action Relationships Supervision (ARS) module enhances the discrimination across action classes through contrastive learning of single-class action-text pairs and models the semantic…
-
13 Jun 2024 1 repository listed Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 6 where Syntology's instrument failed) · 0 unverified (of 9 harvested samples) · 3 pointer-only (licence)To facilitate research on egocentric and exocentric full-body action understanding, we construct benchmarks on a suite of tasks (i. e., action classification, action localization, cross-view sequence verification…
-
11 Jun 2024 1 repository listed Syntology official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 1 honoured, 0 violated, 6 with no contract checked; 4 where Syntology's instrument failed) · 5 unverified (of 16 harvested samples) · 2 pointer-only (licence)With approximately 285 hours of surgical videos, OphNet is about 20 times larger than the largest existing surgical workflow analysis benchmark.
-
5 Jun 2024 1 repository listedFollowing the taxonomy of context-based, generative learning, and contrastive learning approaches, we make a thorough review and benchmark of existing works and shed light on the future possible directions.
-
FineParser: A Fine-grained Spatio-temporal Action Parser for Human-centric Action Quality Assessment11 May 2024 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples) · 5 pointer-only (licence)We argue that a fine-grained understanding of actions requires the model to perceive and parse actions in both time and space, which is also the key to the credibility and interpretability of the AQA technique.
-
3 Jan 2024 1 repository listedReasoning over sports videos for question answering is an important task with numerous applications, such as player training and information retrieval.
-
1 Jan 2024 1 repository listedFine-grained action analysis in multi-person sports is complex due to athletes' quick movements and intense physical confrontations which result in severe visual obstructions in most scenes.
-
25 Dec 2023 1 repository listedA comprehensive understanding of videos is inseparable from describing the action with its contextual action-object interactions.
-
6 Nov 2023 1 repository listedIn this manner, our framework is able to learn the unified representations of uni-modal or multi-modal skeleton input, which is flexible to different kinds of modality input for robust action understanding in practical…
-
15 Aug 2023 1 repository listed Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample)Based on this idea, we present Memory-and-Anticipation Transformer (MAT), a memory-anticipation-based approach, to address the online action detection and anticipation tasks.
-
18 May 2023 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 2 harvested samples) · 2 pointer-only (licence)Action knowledge involves the understanding of textual, visual, and temporal aspects of actions.
-
1 Oct 2022 1 repository listedThis paves the way for a systematic way of evaluating embodied AI agents that understand grounded actions.
-
Weakly-Supervised Temporal Action Detection for Fine-Grained Videos with Hierarchical Atomic Actions24 Jul 2022 1 repository listedAction understanding has evolved into the era of fine granularity, as most human behaviors in real life have only minor differences.
-
19 Jul 2022 1 repository listedAction Quality Assessment(AQA) is important for action understanding and resolving the task poses unique challenges due to subtle visual differences.
-
28 Apr 2022 1 repository listedIn videos that contain actions performed unintentionally, agents do not achieve their desired goals.
-
26 Mar 2022 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)The generated text prompts are paired with corresponding video clips, and together co-train the text encoder and the video encoder via a contrastive approach.
-
28 Feb 2022 1 repository listedTo that end, we propose to learn exercise-oriented image and video representations from unlabeled samples such that a small dataset annotated by experts suffices for supervised error detection.
-
22 Nov 2021 1 repository listedFor human action understanding, a popular research direction is to analyze short video clips with unambiguous semantic content, such as jumping and drinking.
-
3 Sep 2021 1 repository listedThis leads to poor accuracy when downstream tasks, such as action recognition, depend on pose.
-
15 Aug 2021 1 repository listedFine-grained action recognition is attracting increasing attention due to the emerging demand of specific action understanding in real-world applications, whereas the data of rare fine-grained categories is very limited.
-
21 Jun 2021 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 5 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples) · 7 pointer-only (licence)Hand modeling is critical for immersive VR/AR, action understanding, or human healthcare.
-
11 May 2021 1 repository listed Syntology official (archive's flag): 13 ran · 13 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 0 violated, 11 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified (of 16 harvested samples)However, there remains a lack of studies that extend action composition and leverage multiple viewpoints and multiple modalities of data for representation learning.
-
15 Feb 2021 1 repository listedWe introduce a first of its kind paired win-fail action understanding dataset with samples from the following domains: "General Stunts," "Internet Wins-Fails," "Trick Shots," and "Party Games."
-
14 Dec 2020 1 repository listedThe main reason is that large number of nodes (i. e., video frames) makes GCNs hard to capture and model temporal relations in videos.
-
13 Oct 2020 1 repository listedMany believe that the successes of deep learning on image understanding problems can be replicated in the realm of video understanding.
-
31 Aug 2020 1 repository listed
-
31 Jul 2020 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
17 Apr 2020 1 repository listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 0 where Syntology's instrument failed) · 4 unverified (of 12 harvested samples)
-
12 Apr 2020 1 repository listed
-
19 May 2015 1 repository listed
-
The Role of Video Generation in Enhancing Data-Limited Action Understanding26 May 2025 0 repositories listed
-
PCBEAR: Pose Concept Bottleneck for Explainable Action Recognition17 Apr 2025 0 repositories listed
-
RoboAct-CLIP: Video-Driven Pre-training of Atomic Action Understanding for Robotics2 Apr 2025 0 repositories listed
-
Can DeepSeek Reason Like a Surgeon? An Empirical Evaluation for Vision-Language Understanding in Robotic-Assisted Surgery29 Mar 2025 0 repositories listed
-
ScreenLLM: Stateful Screen Schema for Efficient Action Understanding and Prediction26 Mar 2025 0 repositories listed
-
HAIC: Improving Human Action Understanding and Generation with Better Captions for Multi-modal Large Language Models28 Feb 2025 0 repositories listed
-
HumanOmni: A Large Vision-Speech Language Model for Human-Centric Video Understanding25 Jan 2025 0 repositories listed
-
Heterogeneous Skeleton-Based Action Representation Learning1 Jan 2025 0 repositories listed
-
STPro: Spatial and Temporal Progressive Learning for Weakly Supervised Spatio-Temporal Grounding1 Jan 2025 0 repositories listed
-
About Time: Advances, Challenges, and Outlooks of Action Understanding22 Nov 2024 0 repositories listed
-
16 Sep 2024 0 repositories listed Syntology 8 ran (of which 6 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 1 where Syntology's instrument failed) · 4 unverified (of 12 harvested samples) · 12 pointer-only (licence)
-
CathAction: A Benchmark for Endovascular Intervention Understanding23 Aug 2024 0 repositories listed
-
Probing Fine-Grained Action Understanding and Cross-View Generalization of Foundation Models22 Jul 2024 0 repositories listed
-
Region-aware Image-based Human Action Retrieval with Transformers13 Jul 2024 0 repositories listed
-
VELOCITI: Benchmarking Video-Language Compositional Reasoning with Strict Entailment16 Jun 2024 0 repositories listed
-
The SkatingVerse Workshop & Challenge: Methods and Results27 May 2024 0 repositories listed
-
Social-MAE: Social Masked Autoencoder for Multi-person Motion Representation Learning8 Apr 2024 0 repositories listed
-
24 Mar 2024 0 repositories listed
-
Impact of Large Language Model Assistance on Patients Reading Clinical Notes: A Mixed-Methods Study17 Jan 2024 0 repositories listed
-
Multitask Learning in Minimally Invasive Surgical Vision: A Review16 Jan 2024 0 repositories listed
-
28 Nov 2023 0 repositories listed
-
Kantian Deontology Meets AI Alignment: Towards Morally Grounded Fairness Metrics9 Nov 2023 0 repositories listed
-
Comparing Machines and Children: Using Developmental Psychology Experiments to Assess the Strengths and Weaknesses of LaMDA Responses18 May 2023 0 repositories listed
-
ATTACH Dataset: Annotated Two-Handed Assembly Actions for Human Action Understanding17 Apr 2023 0 repositories listed
-
From Isolated Islands to Pangea: Unifying Semantic Space for Human Action Understanding2 Apr 2023 0 repositories listed
-
mRI: Multi-modal 3D Human Pose Estimation Dataset using mmWave, RGB-D, and Inertial Sensors15 Oct 2022 0 repositories listed
-
Invisible-to-Visible: Privacy-Aware Human Instance Segmentation using Airborne Ultrasound via Collaborative Learning Variational Autoencoder15 Apr 2022 0 repositories listed
-
Self-supervised Discovery of Human Actons from Long Kinematic Videos29 Sep 2021 0 repositories listed
-
16 Jun 2021 0 repositories listed
-
Event-based Timestamp Image Encoding Network for Human Action Recognition and Anticipation12 Apr 2021 0 repositories listed
-
GTA: Global Temporal Attention for Video Action Understanding15 Dec 2020 0 repositories listed
-
2 Nov 2020 0 repositories listed
-
4 Aug 2020 0 repositories listed
-
Explainable robotic systems: Understanding goal-driven actions in a reinforcement learning scenario24 Jun 2020 0 repositories listed
-
20 May 2020 0 repositories listed
-
14 Apr 2020 0 repositories listed
-
1 Oct 2019 0 repositories listed
-
Cortical Mirror-System Activation During Real-Life Game Playing: An Intracranial Electroencephalography (EEG) Study27 Mar 2019 0 repositories listed
-
Scene Understanding for Autonomous Manipulation with Deep Learning23 Mar 2019 0 repositories listed
-
Exploring Uncertainty in Conditional Multi-Modal Retrieval Systems23 Jan 2019 0 repositories listed
-
Theory of Minds: Understanding Behavior in Groups Through Inverse Planning18 Jan 2019 0 repositories listed
-
Weakly Supervised Actor-Action Segmentation via Robust Multi-Task Ranking1 Jul 2017 0 repositories listed
-
Action Understanding with Multiple Classes of Actors27 Apr 2017 0 repositories listed
-
22 Mar 2017 0 repositories listed
-
4 Dec 2016 0 repositories listed
-
Hierarchical Attention Network for Action Recognition in Videos21 Jul 2016 0 repositories listed
-
A Hierarchical Pose-Based Approach to Complex Action Understanding Using Dictionaries of Actionlets and Motion Poselets15 Jun 2016 0 repositories listed
-
DAP3D-Net: Where, What and How Actions Occur in Videos?10 Feb 2016 0 repositories listed
-
1 Jun 2015 0 repositories listed
-
Grasp Type Revisited: A Modern Perspective on a Classical Feature for Vision1 Jun 2015 0 repositories listed
-
Human Action Segmentation With Hierarchical Supervoxel Consistency1 Jun 2015 0 repositories listed
-
An Expressive Deep Model for Human Action Parsing from A Single Image2 Feb 2015 0 repositories listed
-
Compositional Structure Learning for Action Understanding21 Oct 2014 0 repositories listed
Syntology lines on 12 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections