{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/upc-s-speech-translation-system-for-iwslt","title":"End-to-End Speech Translation with Pre-trained Models and Adapters: UPC at IWSLT 2021","arxiv_id":"2105.04512","date":"2021-05-10","proceeding":"ACL (IWSLT) 2021 8","authors":["Gerard I. Gállego","Ioannis Tsiamas","Carlos Escolano","José A. R. Fonollosa","Marta R. Costa-jussà"],"abstract":"This paper describes the submission to the IWSLT 2021 offline speech translation task by the UPC Machine Translation group. The task consists of building a system capable of translating English audio recordings extracted from TED talks into German text. Submitted systems can be either cascade or end-to-end and use a custom or given segmentation. Our submission is an end-to-end speech translation system, which combines pre-trained models (Wav2Vec 2.0 and mBART) with coupling modules between the encoder and decoder, and uses an efficient fine-tuning technique, which trains only 20% of its total parameters. We show that adding an Adapter to the system and pre-training it, can increase the convergence speed and the final result, with which we achieve a BLEU score of 27.3 on the MuST-C test set. Our final model is an ensemble that obtains 28.22 BLEU score on the same set. Our submission also uses a custom segmentation algorithm that employs pre-trained Wav2Vec 2.0 for identifying periods of untranscribable text and can bring improvements of 2.5 to 3 BLEU score on the IWSLT 2019 test set, as compared to the result with the given segmentation.","url_abs":"https://arxiv.org/abs/2105.04512v2","url_pdf":"https://arxiv.org/pdf/2105.04512v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"upc-s-speech-translation-system-for-iwslt","repo_url":"https://github.com/mt-upc/iwslt-2021","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"speech-to-text-translation","task_name":"Speech-to-Text Translation"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/speech-to-text-translation-on-must-c-en-de","task":"Speech-to-Text Translation","dataset":"MuST-C EN->DE","model":"Wav2Vec2.0+mBART+Adaptors","rank_in_archive_order":2,"of":8,"metrics":{"Case-sensitive sacreBLEU":"28.22"},"uses_additional_data":true}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2105.04512","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}