Papers › PERF-Net: Pose Empowered RGB-Flow Net
PERF-Net: Pose Empowered RGB-Flow Net
Yinxiao Li, Zhichao Lu, Xuehan Xiong, Jonathan Huang
In recent years, many works in the video action recognition literature have shown that two stream models (combining spatial and temporal input streams) are necessary for achieving state of the art performance. In this paper we show the benefits of including yet another stream based on human pose estimated from each frame -- specifically by rendering pose on input RGB frames. At first blush, this additional stream may seem redundant given that human pose is fully determined by RGB pixel values -- however we show (perhaps surprisingly) that this simple and flexible addition can provide complementary gains. Using this insight, we then propose a new model, which we dub PERF-Net (short for Pose Empowered RGB-Flow Net), which combines this new pose stream with the standard RGB and flow based input streams via distillation techniques and show that our model outperforms the state-of-the-art by a large margin in a number of human action recognition datasets while not requiring flow or pose to be explicitly computed at inference time. The proposed pose stream is also part of the winner solution of the ActivityNet Kinetics Challenge 2020.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Action Classification | Kinetics-600 | PERF-Net (distilled ResNet50-G) | Top-1 Accuracy | 82.0 | #45 of 65 | Archive leaderboard | report |
| Action Classification | Kinetics-600 | PERF-Net (distilled ResNet50-G) | Top-5 Accuracy | 95.7 | #45 of 65 | Archive leaderboard | report |
| Action Recognition | HMDB-51 | PERF-Net (distilled S3D-G) | Average accuracy of 3 splits | 83.2 | #12 of 77 | Archive leaderboard | report |
| Action Recognition | UCF101 | PERF-Net (multi-distilled S3D) | 3-fold Accuracy | 98.6 | #8 of 91 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections