Papers › HandReader: Advanced Techniques for Efficient Fingerspelling Recognition
HandReader: Advanced Techniques for Efficient Fingerspelling Recognition
Pavel Korotaev, Petr Surovtsev, Alexander Kapitanov, Karina Kvanchiani, Aleksandr Nagaev
Fingerspelling is a significant component of Sign Language (SL), allowing the interpretation of proper names, characterized by fast hand movements during signing. Although previous works on fingerspelling recognition have focused on processing the temporal dimension of videos, there remains room for improving the accuracy of these approaches. This paper introduces HandReader, a group of three architectures designed to address the fingerspelling recognition task. HandReader_(RGB) employs the novel Temporal Shift-Adaptive Module (TSAM) to process RGB features from videos of varying lengths while preserving important sequential information. HandReader_(KP) is built on the proposed Temporal Pose Encoder (TPE) operated on keypoints as tensors. Such keypoints composition in a batch allows the encoder to pass them through 2D and 3D convolution layers, utilizing temporal and spatial information and accumulating keypoints coordinates. We also introduce HandReader_RGB+KP - architecture with a joint encoder to benefit from RGB and keypoint modalities. Each HandReader model possesses distinct advantages and achieves state-of-the-art results on the ChicagoFSWild and ChicagoFSWild+ datasets. Moreover, the models demonstrate high performance on the first open dataset for Russian fingerspelling, Znaki, presented in this paper. The Znaki dataset and HandReader pre-trained models are publicly available.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Datasets
Introduced by this paper, per the archive.
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Sign Language Recognition | ChicagoFSWild | HandReader_RGB_KP | CER (%) | 27.1 | #1 of 3 | Archive leaderboard | report |
| Sign Language Recognition | ChicagoFSWild | HandReader_KP | CER (%) | 28 | #2 of 3 | Archive leaderboard | report |
| Sign Language Recognition | ChicagoFSWild | HandReader_RGB | CER (%) | 30.7 | #3 of 3 | Archive leaderboard | report |
| Sign Language Recognition | ChicagoFSWild+ | HandReader_RGB+KP | CER (%) | 24.4 | #1 of 3 | Archive leaderboard | report |
| Sign Language Recognition | ChicagoFSWild+ | HandReader_KP | CER (%) | 26.2 | #2 of 3 | Archive leaderboard | report |
| Sign Language Recognition | ChicagoFSWild+ | HandReader_RGB | CER (%) | 27.6 | #3 of 3 | Archive leaderboard | report |
| Sign Language Recognition | Znaki | HandReader_RGB_KP | CER (%) | 5.06 | #1 of 3 | Archive leaderboard | report |
| Sign Language Recognition | Znaki | HandReader_KP | CER (%) | 7.35 | #2 of 3 | Archive leaderboard | report |
| Sign Language Recognition | Znaki | HandReader_RGB | CER (%) | 7.61 | #3 of 3 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections