Papers › DVANet: Disentangling View and Action Features for Multi-View Action Recognition
DVANet: Disentangling View and Action Features for Multi-View Action Recognition
Nyle Siddiqui, Praveen Tirupattur, Mubarak Shah
In this work, we present a novel approach to multi-view action recognition where we guide learned action representations to be separated from view-relevant information in a video. When trying to classify action instances captured from multiple viewpoints, there is a higher degree of difficulty due to the difference in background, occlusion, and visibility of the captured action from different camera angles. To tackle the various problems introduced in multi-view action recognition, we propose a novel configuration of learnable transformer decoder queries, in conjunction with two supervised contrastive losses, to enforce the learning of action features that are robust to shifts in viewpoints. Our disentangled feature learning occurs in two stages: the transformer decoder uses separate queries to separately learn action and view information, which are then further disentangled using our two contrastive losses. We show that our model and method of training significantly outperforms all other uni-modal models on four multi-view action recognition datasets: NTU RGB+D, NTU RGB+D 120, PKU-MMD, and N-UCLA. Compared to previous RGB works, we see maximal improvements of 1.5\%, 4.8\%, 2.2\%, and 4.8\% on each dataset, respectively.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Action Recognition | N-UCLA | DVANet | Accuracy (Cross-Subject) | 94.4 | #1 of 1 | Archive leaderboard | report |
| Action Recognition | N-UCLA | DVANet | Accuracy (Cross-View) | 96.5 | #1 of 1 | Archive leaderboard | report |
| Action Recognition | NTU RGB+D | DVANet (RGB only) | Accuracy (CS) | 93.4 | #15 of 28 | Archive leaderboard | report |
| Action Recognition | NTU RGB+D | DVANet (RGB only) | Accuracy (CV) | 98.1 | #15 of 28 | Archive leaderboard | report |
| Action Recognition | NTU RGB+D 120 | DVANet (RGB only) | Accuracy (Cross-Setup) | 90.4 | #14 of 21 | Archive leaderboard | report |
| Action Recognition | NTU RGB+D 120 | DVANet (RGB only) | Accuracy (Cross-Subject) | 91.6 | #14 of 21 | Archive leaderboard | report |
| Action Recognition In Videos | PKU-MMD | DVANet (RGB only) | X-Sub | 95.8 | #5 of 5 | Archive leaderboard | report |
| Action Recognition In Videos | PKU-MMD | DVANet (RGB only) | X-View | 95.2 | #5 of 5 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections