Papers › D3D: Distilled 3D Networks for Video Action Recognition
D3D: Distilled 3D Networks for Video Action Recognition
Jonathan C. Stroud, David A. Ross, Chen Sun, Jia Deng, Rahul Sukthankar
State-of-the-art methods for video action recognition commonly use an ensemble of two networks: the spatial stream, which takes RGB frames as input, and the temporal stream, which takes optical flow as input. In recent work, both of these streams consist of 3D Convolutional Neural Networks, which apply spatiotemporal filters to the video clip before performing classification. Conceptually, the temporal filters should allow the spatial stream to learn motion representations, making the temporal stream redundant. However, we still see significant benefits in action recognition performance by including an entirely separate temporal stream, indicating that the spatial stream is "missing" some of the signal captured by the temporal stream. In this work, we first investigate whether motion representations are indeed missing in the spatial stream of 3D CNNs. Second, we demonstrate that these motion representations can be improved by distillation, by tuning the spatial stream to predict the outputs of the temporal stream, effectively combining both models into a single stream. Finally, we show that our Distilled 3D Network (D3D) achieves performance on par with two-stream approaches, using only a single model and with no need to compute optical flow.
In Syntology View this paper on Syntology: its repositories, every harvested function with whether it ran, its licence and the call to fetch it.
Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Action Classification | Kinetics-400 | D3D+S3D-G (RGB + RGB) | Acc@1 | 76.5 | #147 of 207 | Archive leaderboard | report |
| Action Classification | Kinetics-400 | D3D (RGB) | Acc@1 | 75.9 | #153 of 207 | Archive leaderboard | report |
| Action Classification | Kinetics-600 | D3D+S3D-G | Top-1 Accuracy | 79.1 | #53 of 65 | Archive leaderboard | report |
| Action Classification | Kinetics-600 | D3D | Top-1 Accuracy | 77.9 | #56 of 65 | Archive leaderboard | report |
| Action Recognition | AVA v2.1 | D3D (ResNet RPN, Kinetics-400 pretraining) | mAP (Val) | 23 | #11 of 15 | Archive leaderboard | report |
| Action Recognition | HMDB-51 | D3D + D3D | Average accuracy of 3 splits | 80.5 | #23 of 77 | Archive leaderboard | report |
| Action Recognition | HMDB-51 | D3D (Kinetics-600 pretraining) | Average accuracy of 3 splits | 79.3 | #25 of 77 | Archive leaderboard | report |
| Action Recognition | HMDB-51 | D3D (Kinetics-400 pretraining) | Average accuracy of 3 splits | 78.7 | #29 of 77 | Archive leaderboard | report |
| Action Recognition | UCF101 | D3D + D3D | 3-fold Accuracy | 97.6 | #17 of 91 | Archive leaderboard | report |
| Action Recognition | UCF101 | D3D (Kinetics-600 pretraining) | 3-fold Accuracy | 97.1 | #24 of 91 | Archive leaderboard | report |
| Action Recognition | UCF101 | D3D (Kinetics-400 pretraining) | 3-fold Accuracy | 97 | #26 of 91 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections