Papers › PoTion: Pose MoTion Representation for Action Recognition
PoTion: Pose MoTion Representation for Action Recognition
Vasileios Choutas, Philippe Weinzaepfel, Jérôme Revaud, Cordelia Schmid
Most state-of-the-art methods for action recognition rely on a two-stream architecture that processes appearance and motion independently. In this paper, we claim that considering them jointly offers rich information for action recognition. We introduce a novel representation that gracefully encodes the movement of some semantic keypoints. We use the human joints as these keypoints and term our Pose moTion representation PoTion. Specifically, we first run a state-of-the-art human pose estimator and extract heatmaps for the human joints in each frame. We obtain our PoTion representation by temporally aggregating these probability maps. This is achieved by colorizing each of them depending on the relative time of the frames in the video clip and summing them. This fixed-size representation for an entire video clip is suitable to classify actions using a shallow convolutional neural network. Our experimental evaluation shows that PoTion outperforms other state-of-the-art pose representations. Furthermore, it is complementary to standard appearance and motion streams. When combining PoTion with the recent two-stream I3D approach [5], we obtain state-of-the-art performance on the JHMDB, HMDB and UCF101 datasets.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Action Classification | Charades | PoTion + (GCN + I3D + NL I3D) | MAP | 40.8 | #32 of 49 | Archive leaderboard | report |
| Action Recognition | UCF101 | I3D + PoTion | 3-fold Accuracy | 29.3 | #89 of 91 | Archive leaderboard | report |
| Skeleton Based Action Recognition | J-HMDB | Potion | Accuracy (RGB+pose) | 90.4 | #1 of 13 | Archive leaderboard | report |
| Skeleton Based Action Recognition | J-HMDB | Potion | Accuracy (pose) | 67.9 | #1 of 13 | Archive leaderboard | report |
| Skeleton Based Action Recognition | J-HMDB | I3D + Potion | Accuracy (RGB+pose) | 85.5 | #3 of 13 | Archive leaderboard | report |
| Skeleton Based Action Recognition | JHMDB (2D poses only) | PoTion | Average accuracy of 3 splits | 67.9 | #4 of 6 | Archive leaderboard | report |
| Skeleton Based Action Recognition | JHMDB (2D poses only) | PoTion | No. parameters | - | #4 of 6 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections