Speech-to-Text Translation

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

11 leaderboard tables shown for this task, 11 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted. 10 shown of 11 until expanded.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

4 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

1 subtask in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 64 papers with code (146 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• fairseq S2T: Fast Speech-to-Text Modeling with fairseq
  11 Oct 2020 5 repositories listed
  We introduce fairseq S2T, a fairseq extension for speech-to-text (S2T) modeling tasks such as end-to-end speech recognition and speech-to-text translation.
• SeamlessM4T: Massively Multilingual & Multimodal Machine Translation
  22 Aug 2023 4 repositories listed Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
  What does it take to create the Babel Fish, a tool that can help individuals translate speech between any two languages?
• LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPT
  7 Oct 2023 2 repositories listed Syntology ran 3 of 3 samples · 0 unverified
  Previous mainstream audio-and-text LLMs use discrete audio tokens to represent both input and output audio; however, they suffer from performance degradation on tasks such as automatic speech recognition, speech-to-text…
• PaddleSpeech: An Easy-to-Use All-in-One Speech Toolkit
  20 May 2022 2 repositories listed
  PaddleSpeech is an open-source all-in-one speech toolkit.
• A³T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editing
  18 Mar 2022 2 repositories listed Syntology ran 0 of 7 samples · 7 unverified
  Recently, speech representation learning has improved many speech-related tasks such as speech recognition, speech classification, and speech-to-text translation.
• SHAS: Approaching optimal Segmentation for End-to-End Speech Translation
  9 Feb 2022 2 repositories listed
  Speech translation datasets provide manual segmentations of the audios, which are not available in real-world scenarios, and existing segmentation methods usually significantly reduce translation quality at inference…
• Lightweight Adapter Tuning for Multilingual Speech Translation
  2 Jun 2021 2 repositories listed
  Adapter modules were recently introduced as an efficient alternative to fine-tuning in NLP.
• Learning Shared Semantic Space for Speech-to-Text Translation
  7 May 2021 2 repositories listed Syntology ran 0 of 7 samples · 7 unverified · 7 pointer-only (licence)
  By projecting audio and text features to a common semantic representation, Chimera unifies MT and ST tasks and boosts the performance on ST benchmarks, MuST-C and Augmented Librispeech, to a new state-of-the-art.
• CoVoST 2 and Massively Multilingual Speech-to-Text Translation
  20 Jul 2020 2 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 3 pointer-only (licence)
  Speech translation has recently become an increasingly popular topic of research, partly due to the development of benchmark datasets.
• BeaverTalk: Oregon State University's IWSLT 2025 Simultaneous Speech Translation System
  29 May 2025 1 repository listed
  In particular, on the English→German task, the system achieves a BLEU of 24.
• Audio Jailbreak Attacks: Exposing Vulnerabilities in SpeechGPT in a White-Box Framework
  24 May 2025 1 repository listed
  Recent advances in Multimodal Large Language Models (MLLMs) have significantly enhanced the naturalness and flexibility of human computer interaction by enabling seamless understanding across text, vision, and audio…
• MEDIBENG WHISPER TINY: A FINE-TUNED CODE-SWITCHED BENGALI-ENGLISH TRANSLATOR FOR CLINICAL APPLICATIONS
  25 Apr 2025 1 repository listed
  The model was fine-tuned on MediBeng, a synthetic dataset created to simulate the types of bilingual interactions often found in healthcare environments.
• SparQLe: Speech Queries to Text Translation Through LLMs
  13 Feb 2025 1 repository listed
  With the growing influence of Large Language Models (LLMs), there is increasing interest in integrating speech representations with them to enable more seamless multi-modal processing and speech understanding.
• Fleurs-SLU: A Massively Multilingual Benchmark for Spoken Language Understanding
  10 Jan 2025 1 repository listed
  While recent multilingual automatic speech recognition models claim to support thousands of languages, ASR for low-resource languages remains highly unreliable due to limited bimodal speech and text training data.
• LLaST: Improved End-to-end Speech Translation System Leveraged by Large Language Models
  22 Jul 2024 1 repository listed
  We introduces LLaST, a framework for building high-performance Large Language model based Speech-to-text Translation systems.
• CoVoSwitch: Machine Translation of Synthetic Code-Switched Text Based on Intonation Units
  19 Jul 2024 1 repository listed
  We reveal that the inclusion of code-switching units results in higher translation performance than monolingual settings and that models are better at code-switching translation into English than non-English.
• Listen and Speak Fairly: A Study on Semantic Gender Bias in Speech Integrated Large Language Models
  9 Jul 2024 1 repository listed
  Speech Integrated Large Language Models (SILLMs) combine large language models with speech perception to perform diverse tasks, such as emotion recognition to speaker verification, demonstrating universal audio…
• Voices Unheard: NLP Resources and Models for Yorùbá Regional Dialects
  27 Jun 2024 1 repository listed
  Recent efforts to develop NLP technologies for African languages have focused on their standard dialects, resulting in disparities for dialects and varieties for which there are little to no resources or tools.
• ArzEn-LLM: Code-Switched Egyptian Arabic-English Translation and Speech Recognition Using LLMs
  26 Jun 2024 1 repository listed
  Our models and code are available as open-source resources.
• SimulSeamless: FBK at IWSLT 2024 Simultaneous Speech Translation
  20 Jun 2024 1 repository listed
  This paper describes the FBK's participation in the Simultaneous Translation Evaluation Campaign at IWSLT 2024.
• StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selection
  10 Jun 2024 1 repository listed
  To fill this gap, we introduce StreamAtt, the first StreamST policy, and propose StreamLAAL, the first StreamST latency metric designed to be comparable with existing metrics for SimulST.
• StreamSpeech: Simultaneous Speech-to-Speech Translation with Multi-task Learning
  5 Jun 2024 1 repository listed Syntology ran 4 of 4 samples · 0 unverified
  Simultaneous speech-to-speech translation (Simul-S2ST, a.
• LeaPformer: Enabling Linear Transformers for Autoregressive and Simultaneous Tasks via Learned Proportions
  18 May 2024 1 repository listed Syntology ran 2 of 4 samples · 2 unverified
  A promising approach to preserving model performance in linearized transformers is to employ position-based re-weighting functions.
• Pushing the Limits of Zero-shot End-to-End Speech Translation
  16 Feb 2024 1 repository listed
  The speech encoder seamlessly integrates with the MT model at inference, enabling direct translation from speech to text, across all languages supported by the MT model.
• Investigating Zero-Shot Generalizability on Mandarin-English Code-Switched ASR and Speech-to-text Translation of Recent Foundation Models with Self-Supervision and Weak Supervision
  30 Dec 2023 1 repository listed
  This work evaluated several cutting-edge large-scale foundation models based on self-supervision or weak supervision, including SeamlessM4T, SeamlessM4T v2, and Whisper-large-v3, on three code-switched corpora.
• End-to-End Single-Channel Speaker-Turn Aware Conversational Speech Translation
  1 Nov 2023 1 repository listed
  Conventional speech-to-text translation (ST) systems are trained on single-speaker utterances, and they may not generalize to real-life scenarios where the audio contains conversations by multiple speakers.
• An Empirical Study of Consistency Regularization for End-to-End Speech-to-Text Translation
  28 Aug 2023 1 repository listed
  Consistency regularization methods, such as R-Drop (Liang et al., 2021) and CrossConST (Gao et al., 2023), have achieved impressive supervised and zero-shot performance in the neural machine translation (NMT) field.
• SONAR: Sentence-Level Multimodal and Language-Agnostic Representations
  22 Aug 2023 1 repository listed
  Our single text encoder, covering 200 languages, substantially outperforms existing sentence embeddings such as LASER3 and LabSE on the xsim and xsim++ multilingual similarity search tasks.
• ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation
  24 May 2023 1 repository listed
  Joint speech-language training is challenging due to the large demand for training data and GPU consumption, as well as the modality gap between speech and language.
• DUB: Discrete Unit Back-translation for Speech Translation
  19 May 2023 1 repository listed
  The key point is to bridge the modality gap between speech and text so that useful MT techniques can be applied to ST.

Syntology lines on 7 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.