Papers › STAR-Transformer: A Spatio-temporal Cross Attention Transformer for Human Action Recognition
STAR-Transformer: A Spatio-temporal Cross Attention Transformer for Human Action Recognition
Dasom Ahn, Sangwon Kim, Hyunsu Hong, Byoung Chul Ko
In action recognition, although the combination of spatio-temporal videos and skeleton features can improve the recognition performance, a separate model and balancing feature representation for cross-modal data are required. To solve these problems, we propose Spatio-TemporAl cRoss (STAR)-transformer, which can effectively represent two cross-modal features as a recognizable vector. First, from the input video and skeleton sequence, video frames are output as global grid tokens and skeletons are output as joint map tokens, respectively. These tokens are then aggregated into multi-class tokens and input into STAR-transformer. The STAR-transformer encoder layer consists of a full self-attention (FAttn) module and a proposed zigzag spatio-temporal attention (ZAttn) module. Similarly, the continuous decoder consists of a FAttn module and a proposed binary spatio-temporal attention (BAttn) module. STAR-transformer learns an efficient multi-feature representation of the spatio-temporal features by properly arranging pairings of the FAttn, ZAttn, and BAttn modules. Experimental results on the Penn-Action, NTU RGB+D 60, and 120 datasets show that the proposed method achieves a promising improvement in performance in comparison to previous state-of-the-art methods.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Action Recognition | NTU RGB+D | STAR-Transformer (RGB + Pose) | Accuracy (CS) | 92.0 | #20 of 28 | Archive leaderboard | report |
| Action Recognition | NTU RGB+D | STAR-Transformer (RGB + Pose) | Accuracy (CV) | 96.5 | #20 of 28 | Archive leaderboard | report |
| Action Recognition | NTU RGB+D 120 | STAR-Transformer (RGB + Pose) | Accuracy (Cross-Setup) | 92.7 | #6 of 21 | Archive leaderboard | report |
| Action Recognition | NTU RGB+D 120 | STAR-Transformer (RGB + Pose) | Accuracy (Cross-Subject) | 90.3 | #6 of 21 | Archive leaderboard | report |
| Action Recognition | Penn Action | STAR-Transformer (RGB + Pose) | Accuracy | 98.7 | #2 of 2 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections