{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-non-autoregressive-generation-framework-for","title":"A Non-autoregressive Generation Framework for End-to-End Simultaneous Speech-to-Speech Translation","arxiv_id":"2406.06937","date":"2024-06-11","proceeding":null,"authors":["Zhengrui Ma","Qingkai Fang","Shaolei Zhang","Shoutao Guo","Yang Feng","Min Zhang"],"abstract":"Simultaneous translation models play a crucial role in facilitating communication. However, existing research primarily focuses on text-to-text or speech-to-text models, necessitating additional cascade components to achieve speech-to-speech translation. These pipeline methods suffer from error propagation and accumulate delays in each cascade component, resulting in reduced synchronization between the speaker and listener. To overcome these challenges, we propose a novel non-autoregressive generation framework for simultaneous speech translation (NAST-S2X), which integrates speech-to-text and speech-to-speech tasks into a unified end-to-end framework. We develop a non-autoregressive decoder capable of concurrently generating multiple text or acoustic unit tokens upon receiving fixed-length speech chunks. The decoder can generate blank or repeated tokens and employ CTC decoding to dynamically adjust its latency. Experimental results show that NAST-S2X outperforms state-of-the-art models in both speech-to-text and speech-to-speech tasks. It achieves high-quality simultaneous interpretation within a delay of less than 3 seconds and provides a 28 times decoding speedup in offline generation.","url_abs":"https://arxiv.org/abs/2406.06937v2","url_pdf":"https://arxiv.org/pdf/2406.06937v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-non-autoregressive-generation-framework-for","repo_url":"https://github.com/ictnlp/nast-s2x","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"simultaneous-speech-to-speech-translation","task_name":"Simultaneous Speech-to-Speech Translation"},{"task_slug":"speech-to-speech-translation","task_name":"Speech-to-Speech Translation"},{"task_slug":"speech-to-text","task_name":"Speech-to-Text"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2406.06937","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.06937"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ictnlp/nast-s2x","reach":{"status":"ok"}}],"summary":{"ran":2},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"9ad8e93984675d06","entry":"gen_block_atten_mask","repo":"ictnlp/nast-s2x","repo_kind":"official","path":"nast/modules/unidirectional_encoder.py","file_url":"https://github.com/ictnlp/nast-s2x/blob/HEAD/nast/modules/unidirectional_encoder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9ad8e93984675d06"}},{"code_sha256_prefix":"9fb1e779a2221582","entry":"lengths_to_padding_mask","repo":"ictnlp/nast-s2x","repo_kind":"official","path":"nast/modules/audio_convs.py","file_url":"https://github.com/ictnlp/nast-s2x/blob/HEAD/nast/modules/audio_convs.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9fb1e779a2221582"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}