{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gloss-attention-for-gloss-free-sign-language-1","title":"Gloss Attention for Gloss-free Sign Language Translation","arxiv_id":"2307.07361","date":"2023-07-14","proceeding":"CVPR 2023 1","authors":["Aoxiong Yin","Tianyun Zhong","Li Tang","Weike Jin","Tao Jin","Zhou Zhao"],"abstract":"Most sign language translation (SLT) methods to date require the use of gloss annotations to provide additional supervision information, however, the acquisition of gloss is not easy. To solve this problem, we first perform an analysis of existing models to confirm how gloss annotations make SLT easier. We find that it can provide two aspects of information for the model, 1) it can help the model implicitly learn the location of semantic boundaries in continuous sign language videos, 2) it can help the model understand the sign language video globally. We then propose \\emph{gloss attention}, which enables the model to keep its attention within video segments that have the same semantics locally, just as gloss helps existing models do. Furthermore, we transfer the knowledge of sentence-to-sentence similarity from the natural language model to our gloss attention SLT network (GASLT) to help it understand sign language videos at the sentence level. Experimental results on multiple large-scale sign language datasets show that our proposed GASLT model significantly outperforms existing methods. Our code is provided in \\url{https://github.com/YinAoXiong/GASLT}.","url_abs":"https://arxiv.org/abs/2307.07361v1","url_pdf":"https://arxiv.org/pdf/2307.07361v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"gloss-attention-for-gloss-free-sign-language-1","repo_url":"https://github.com/yinaoxiong/gaslt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"gloss-free-sign-language-translation","task_name":"Gloss-free Sign Language Translation"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"sentence-similarity","task_name":"Sentence Similarity"},{"task_slug":"sign-language-translation","task_name":"Sign Language Translation"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2307.07361","atlas_url":"https://app.syntology.ai/?focus=2307.07361","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.07361"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/tylin/coco-caption","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yinaoxiong/gaslt","reach":null}],"summary":{"ran":2},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"f04b54669924fe0f","entry":"AttentionMechanism","repo":"yinaoxiong/gaslt","repo_kind":"official","path":"signjoey/attention.py","file_url":"https://github.com/yinaoxiong/gaslt/blob/HEAD/signjoey/attention.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f04b54669924fe0f"}},{"code_sha256_prefix":"8b2573fc88ba1f67","entry":"BahdanauAttention","repo":"yinaoxiong/gaslt","repo_kind":"official","path":"signjoey/attention.py","file_url":"https://github.com/yinaoxiong/gaslt/blob/HEAD/signjoey/attention.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8b2573fc88ba1f67"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}