Papers › StepNet: Spatial-temporal Part-aware Network for Isolated Sign Language Recognition
StepNet: Spatial-temporal Part-aware Network for Isolated Sign Language Recognition
Xiaolong Shen, Zhedong Zheng, Yi Yang
The goal of sign language recognition (SLR) is to help those who are hard of hearing or deaf overcome the communication barrier. Most existing approaches can be typically divided into two lines, i.e., Skeleton-based and RGB-based methods, but both the two lines of methods have their limitations. Skeleton-based methods do not consider facial expressions, while RGB-based approaches usually ignore the fine-grained hand structure. To overcome both limitations, we propose a new framework called Spatial-temporal Part-aware network~(StepNet), based on RGB parts. As its name suggests, it is made up of two modules: Part-level Spatial Modeling and Part-level Temporal Modeling. Part-level Spatial Modeling, in particular, automatically captures the appearance-based properties, such as hands and faces, in the feature space without the use of any keypoint-level annotations. On the other hand, Part-level Temporal Modeling implicitly mines the long-short term context to capture the relevant attributes over time. Extensive experiments demonstrate that our StepNet, thanks to spatial-temporal modules, achieves competitive Top-1 Per-instance accuracy on three commonly-used SLR benchmarks, i.e., 56.89% on WLASL, 77.2% on NMFs-CSL, and 77.1% on BOBSL. Additionally, the proposed method is compatible with the optical flow input and can produce superior performance if fused. For those who are hard of hearing, we hope that our work can act as a preliminary step.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Sign Language Recognition | BOBSL | StepNet | Actions Top-1 | 77.1 | #1 of 1 | Archive leaderboard | report |
| Sign Language Recognition | WLASL | StepNet | Top-1 Accuracy | 61.17 | #1 of 2 | Archive leaderboard | report |
| Sign Language Recognition | WLASL-2000 | StepNet | Top-1 Accuracy | 61.17 | #4 of 9 | Archive leaderboard | report |
| Sign Language Recognition | WLASL100 | StepNet | Official Test Split | true | #5 of 7 | Archive leaderboard | report |
| Sign Language Recognition | WLASL100 | StepNet | Top-1 Accuracy | 78.29 | #5 of 7 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections