{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/end-to-end-asr-from-supervised-to-semi","title":"End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures","arxiv_id":"1911.08460","date":"2019-11-19","proceeding":null,"authors":["Gabriel Synnaeve","Qiantong Xu","Jacob Kahn","Tatiana Likhomanenko","Edouard Grave","Vineel Pratap","Anuroop Sriram","Vitaliy Liptchinsky","Ronan Collobert"],"abstract":"We study pseudo-labeling for the semi-supervised training of ResNet, Time-Depth Separable ConvNets, and Transformers for speech recognition, with either CTC or Seq2Seq loss functions. We perform experiments on the standard LibriSpeech dataset, and leverage additional unlabeled data from LibriVox through pseudo-labeling. We show that while Transformer-based acoustic models have superior performance with the supervised dataset alone, semi-supervision improves all models across architectures and loss functions and bridges much of the performance gaps between them. In doing so, we reach a new state-of-the-art for end-to-end acoustic models decoded with an external language model in the standard supervised learning setting, and a new absolute state-of-the-art with semi-supervised training. Finally, we study the effect of leveraging different amounts of unlabeled audio, propose several ways of evaluating the characteristics of unlabeled audio which improve acoustic modeling, and show that acoustic models trained with more audio rely less on external language models.","url_abs":"https://arxiv.org/abs/1911.08460v3","url_pdf":"https://arxiv.org/pdf/1911.08460v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"end-to-end-asr-from-supervised-to-semi","repo_url":"https://github.com/facebookresearch/wav2letter/tree/master/recipes/models/sota/2019","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"speech-recognition","task_name":"Speech Recognition"},{"task_slug":"speech-recognition-1","task_name":"speech-recognition"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"seq2seq","method_name":"Seq2Seq"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/speech-recognition-on-librispeech-test-clean","task":"Speech Recognition","dataset":"LibriSpeech test-clean","model":"Conv + Transformer AM + Pseudo-Labeling (ConvLM with Transformer Rescoring)","rank_in_archive_order":27,"of":64,"metrics":{"Word Error Rate (WER)":"2.03"},"uses_additional_data":false},{"leaderboard":"/sota/speech-recognition-on-librispeech-test-clean","task":"Speech Recognition","dataset":"LibriSpeech test-clean","model":"Conv + Transformer AM (ConvLM  with Transformer Rescoring) (LS only)","rank_in_archive_order":37,"of":64,"metrics":{"Word Error Rate (WER)":"2.31"},"uses_additional_data":false},{"leaderboard":"/sota/speech-recognition-on-librispeech-test-other","task":"Speech Recognition","dataset":"LibriSpeech test-other","model":"Conv + Transformer AM (ConvLM with Transformer Rescoring)","rank_in_archive_order":19,"of":53,"metrics":{"Word Error Rate (WER)":"4.11"},"uses_additional_data":true},{"leaderboard":"/sota/speech-recognition-on-librispeech-test-other","task":"Speech Recognition","dataset":"LibriSpeech test-other","model":"Conv + Transformer AM (ConvLM  with Transformer Rescoring) (LS only)","rank_in_archive_order":32,"of":53,"metrics":{"Word Error Rate (WER)":"5.18"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1911.08460","atlas_url":"https://app.syntology.ai/?focus=1911.08460","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}