Papers › Pose And Joint-Aware Action Recognition
Pose And Joint-Aware Action Recognition
Anshul Shah, Shlok Mishra, Ankan Bansal, Jun-Cheng Chen, Rama Chellappa, Abhinav Shrivastava
Recent progress on action recognition has mainly focused on RGB and optical flow features. In this paper, we approach the problem of joint-based action recognition. Unlike other modalities, constellation of joints and their motion generate models with succinct human motion information for activity recognition. We present a new model for joint-based action recognition, which first extracts motion features from each joint separately through a shared motion encoder before performing collective reasoning. Our joint selector module re-weights the joint information to select the most discriminative joints for the task. We also propose a novel joint-contrastive loss that pulls together groups of joint features which convey the same action. We strengthen the joint-based representations by using a geometry-aware data augmentation technique which jitters pose heatmaps while retaining the dynamics of the action. We show large improvements over the current state-of-the-art joint-based approaches on JHMDB, HMDB, Charades, AVA action recognition datasets. A late fusion with RGB and Flow-based approaches yields additional improvements. Our model also outperforms the existing baseline on Mimetics, a dataset with out-of-context actions.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
1 archive task tag without a task page not shown.
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Action Classification | Charades | JMRN + R101-NL-LFB | MAP | 43.23 | #23 of 49 | Archive leaderboard | report |
| Action Classification | Charades | JMRN (Pose only) | MAP | 16.2 | #49 of 49 | Archive leaderboard | report |
| Action Recognition | AVA v2.1 | JMRN + SlowFast-R101-NL | mAP (Val) | 28.4 | #3 of 15 | Archive leaderboard | report |
| Action Recognition | HMDB-51 | Ours + ResNext101 BERT | Average accuracy of 3 splits | 84.53 | #8 of 77 | Archive leaderboard | report |
| Action Recognition | HMDB-51 | JRMN | Average accuracy of 3 splits | 54.2 | #73 of 77 | Archive leaderboard | report |
| Action Recognition | Mimetics | JMRN | mAP | 40 | #1 of 2 | Archive leaderboard | report |
| Action Recognition | Mimetics | SIP-Net | mAP | 38.3 | #2 of 2 | Archive leaderboard | report |
| Skeleton Based Action Recognition | JHMDB (2D poses only) | JMRN (No GT pose) | Average accuracy of 3 splits | 68.55 | #3 of 6 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections