Papers › Improving Sign Language Translation with Monolingual Data by Sign Back-Translation
Improving Sign Language Translation with Monolingual Data by Sign Back-Translation
Hao Zhou, Wengang Zhou, Weizhen Qi, Junfu Pu, Houqiang Li
Despite existing pioneering works on sign language translation (SLT), there is a non-trivial obstacle, i.e., the limited quantity of parallel sign-text data. To tackle this parallel data bottleneck, we propose a sign back-translation (SignBT) approach, which incorporates massive spoken language texts into SLT training. With a text-to-gloss translation model, we first back-translate the monolingual text to its gloss sequence. Then, the paired sign sequence is generated by splicing pieces from an estimated gloss-to-sign bank at the feature level. Finally, the synthetic parallel data serves as a strong supplement for the end-to-end training of the encoder-decoder SLT framework. To promote the SLT research, we further contribute CSL-Daily, a large-scale continuous SLT dataset. It provides both spoken language translations and gloss-level annotations. The topic revolves around people's daily lives (e.g., travel, shopping, medical care), the most likely SLT application scenario. Extensive experimental results and analysis of SLT methods are reported on CSL-Daily. With the proposed sign back-translation method, we obtain a substantial improvement over previous state-of-the-art SLT methods.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Datasets
Introduced by this paper, per the archive.
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Sign Language Recognition | CSL-Daily | BN-TIN+Transf. | Word Error Rate (WER) | 33.1 | #14 of 14 | Archive leaderboard | report |
| Sign Language Recognition | RWTH-PHOENIX-Weather 2014 T | SignBT | Word Error Rate (WER) | 23.9 | #13 of 15 | Archive leaderboard | report |
| Sign Language Translation | CSL-Daily | BN-TIN-Transf.+SignBT | BLEU-4 | 21.34 | #5 of 8 | Archive leaderboard | report |
| Sign Language Translation | CSL-Daily | BN-TIN-Transf. | BLEU-4 | 13.19 | #8 of 8 | Archive leaderboard | report |
| Sign Language Translation | RWTH-PHOENIX-Weather 2014 T | BN-TIN-Transf.+SignBT | BLEU-4 | 24.32 | #6 of 11 | Archive leaderboard | report |
| Sign Language Translation | RWTH-PHOENIX-Weather 2014 T | BN-TIN-Transf. | BLEU-4 | 21.68 | #9 of 11 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections