{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/very-deep-multilingual-convolutional-neural","title":"Very Deep Multilingual Convolutional Neural Networks for LVCSR","arxiv_id":"1509.08967","date":"2015-09-29","proceeding":null,"authors":["Tom Sercu","Christian Puhrsch","Brian Kingsbury","Yann Lecun"],"abstract":"Convolutional neural networks (CNNs) are a standard component of many current\nstate-of-the-art Large Vocabulary Continuous Speech Recognition (LVCSR)\nsystems. However, CNNs in LVCSR have not kept pace with recent advances in\nother domains where deeper neural networks provide superior performance. In\nthis paper we propose a number of architectural advances in CNNs for LVCSR.\nFirst, we introduce a very deep convolutional network architecture with up to\n14 weight layers. There are multiple convolutional layers before each pooling\nlayer, with small 3x3 kernels, inspired by the VGG Imagenet 2014 architecture.\nThen, we introduce multilingual CNNs with multiple untied layers. Finally, we\nintroduce multi-scale input features aimed at exploiting more context at\nnegligible computational cost. We evaluate the improvements first on a Babel\ntask for low resource speech recognition, obtaining an absolute 5.77% WER\nimprovement over the baseline PLP DNN by training our CNN on the combined data\nof six different languages. We then evaluate the very deep CNNs on the Hub5'00\nbenchmark (using the 262 hours of SWB-1 training data) achieving a word error\nrate of 11.8% after cross-entropy training, a 1.4% WER improvement (10.6%\nrelative) over the best published CNN result so far.","url_abs":"http://arxiv.org/abs/1509.08967v2","url_pdf":"http://arxiv.org/pdf/1509.08967v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"speech-recognition","task_name":"Speech Recognition"},{"task_slug":"speech-recognition-1","task_name":"speech-recognition"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/speech-recognition-on-switchboard-hub500","task":"Speech Recognition","dataset":"Switchboard + Hub500","model":"Deep CNN (10 conv, 4 FC layers), multi-scale feature maps","rank_in_archive_order":17,"of":30,"metrics":{"Percentage error":"12.2"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1509.08967","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}