Papers › Zero-shot Skeleton-based Action Recognition via Mutual Information Estimation and Maximization
Zero-shot Skeleton-based Action Recognition via Mutual Information Estimation and Maximization
Yujie Zhou, Wenwen Qiang, Anyi Rao, Ning Lin, Bing Su, Jiaqi Wang
Zero-shot skeleton-based action recognition aims to recognize actions of unseen categories after training on data of seen categories. The key is to build the connection between visual and semantic space from seen to unseen classes. Previous studies have primarily focused on encoding sequences into a singular feature vector, with subsequent mapping the features to an identical anchor point within the embedded space. Their performance is hindered by 1) the ignorance of the global visual/semantic distribution alignment, which results in a limitation to capture the true interdependence between the two spaces. 2) the negligence of temporal information since the frame-wise features with rich action clues are directly pooled into a single feature vector. We propose a new zero-shot skeleton-based action recognition method via mutual information (MI) estimation and maximization. Specifically, 1) we maximize the MI between visual and semantic space for distribution alignment; 2) we leverage the temporal information for estimating the MI by encouraging MI to increase as more frames are observed. Extensive experiments on three large-scale skeleton action datasets confirm the effectiveness of our method. Code: https://github.com/YujieOuO/SMIE.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Zero Shot Skeletal Action Recognition | NTU RGB+D | SMIE | Accuracy (12 unseen classes) | 40.18 | #6 of 9 | Archive leaderboard | report |
| Zero Shot Skeletal Action Recognition | NTU RGB+D | SMIE | Accuracy (5 unseen classes) | 77.98 | #6 of 9 | Archive leaderboard | report |
| Zero Shot Skeletal Action Recognition | NTU RGB+D | SMIE | Random Split Accuracy | 65.08 | #6 of 9 | Archive leaderboard | report |
| Zero Shot Skeletal Action Recognition | NTU RGB+D 120 | SMIE | Accuracy (10 unseen classes) | 65.74 | #5 of 9 | Archive leaderboard | report |
| Zero Shot Skeletal Action Recognition | NTU RGB+D 120 | SMIE | Accuracy (24 unseen classes) | 45.30 | #5 of 9 | Archive leaderboard | report |
| Zero Shot Skeletal Action Recognition | NTU RGB+D 120 | SMIE | Random Split Accuracy | 46.40 | #5 of 9 | Archive leaderboard | report |
| Zero Shot Skeletal Action Recognition | PKU-MMD | SMIE | Random Split Accuracy | 60.83 | #5 of 7 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections