Papers › Pose And Joint-Aware Action Recognition
Pose And Joint-Aware Action Recognition
Anshul Shah, Shlok Mishra, Ankan Bansal, Jun-Cheng Chen, Rama Chellappa, Abhinav Shrivastava
Recent progress on action recognition has mainly focused on RGB and optical flow features. In this paper, we approach the problem of joint-based action recognition. Unlike other modalities, constellation of joints and their motion generate models with succinct human motion information for activity recognition. We present a new model for joint-based action recognition, which first extracts motion features from each joint separately through a shared motion encoder before performing collective reasoning. Our joint selector module re-weights the joint information to select the most discriminative joints for the task. We also propose a novel joint-contrastive loss that pulls together groups of joint features which convey the same action. We strengthen the joint-based representations by using a geometry-aware data augmentation technique which jitters pose heatmaps while retaining the dynamics of the action. We show large improvements over the current state-of-the-art joint-based approaches on JHMDB, HMDB, Charades, AVA action recognition datasets. A late fusion with RGB and Flow-based approaches yields additional improvements. Our model also outperforms the existing baseline on Mimetics, a dataset with out-of-context actions.
In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.
Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
For agents, on Syntology's MCP service (how to connect):
get_citation_path(paper_1="2010.08164", paper_2="…")with another paper's arXiv id or titleget_concepts_for_paper(arxiv_id="2010.08164")
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
1 archive task tag without a task page not shown.
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Action Classification | Charades | JMRN + R101-NL-LFB | MAP | 43.23 | #23 of 49 | Archive leaderboard | report |
| Action Classification | Charades | JMRN (Pose only) | MAP | 16.2 | #49 of 49 | Archive leaderboard | report |
| Action Recognition | AVA v2.1 | JMRN + SlowFast-R101-NL | mAP (Val) | 28.4 | #3 of 15 | Archive leaderboard | report |
| Action Recognition | HMDB-51 | Ours + ResNext101 BERT | Average accuracy of 3 splits | 84.53 | #8 of 77 | Archive leaderboard | report |
| Action Recognition | HMDB-51 | JRMN | Average accuracy of 3 splits | 54.2 | #73 of 77 | Archive leaderboard | report |
| Action Recognition | Mimetics | JMRN | mAP | 40 | #1 of 2 | Archive leaderboard | report |
| Action Recognition | Mimetics | SIP-Net | mAP | 38.3 | #2 of 2 | Archive leaderboard | report |
| Skeleton Based Action Recognition | JHMDB (2D poses only) | JMRN (No GT pose) | Average accuracy of 3 splits | 68.55 | #3 of 6 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections