{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/neural-network-language-modeling-with-letter","title":"Neural Network Language Modeling with Letter-based Features and Importance Sampling","arxiv_id":null,"date":"2018-04-15","proceeding":"ICASSP 2018 4","authors":["Hainan Xu","Ke Li","Yiming Wang","Jian Wang","Shiyin Kang","Xie Chen","Daniel Povey","Sanjeev Khudanpur"],"abstract":"In this paper we describe an extension of the Kaldi software toolkit to support neural-based language modeling, intended\r\nfor use in automatic speech recognition (ASR) and related tasks. We combine the use of subword features (letter n-grams) and one-hot encoding of frequent words so that the models can handle large vocabularies containing infrequent\r\nwords. We propose a new objective function that allows for training of unnormalized probabilities. An importance sampling based method is supported to speed up training when the vocabulary is large. Experimental results on five corpora show that Kaldi-RNNLM rivals other recurrent neural network language model toolkits both on performance and training speed.","url_abs":"https://www.cs.jhu.edu/~hxu/neural-network-language.pdf","url_pdf":"https://www.cs.jhu.edu/~hxu/neural-network-language.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"automatic-speech-recognition-2","task_name":"Automatic Speech Recognition"},{"task_slug":"automatic-speech-recognition","task_name":"Automatic Speech Recognition (ASR)"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"speech-recognition","task_name":"Speech Recognition"},{"task_slug":"speech-recognition-1","task_name":"speech-recognition"}],"methods":[{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/speech-recognition-on-librispeech-test-clean","task":"Speech Recognition","dataset":"LibriSpeech test-clean","model":"tdnn + chain + rnnlm rescoring","rank_in_archive_order":47,"of":64,"metrics":{"Word Error Rate (WER)":"3.06"},"uses_additional_data":false},{"leaderboard":"/sota/speech-recognition-on-librispeech-test-other","task":"Speech Recognition","dataset":"LibriSpeech test-other","model":"tdnn + chain + rnnlm rescoring","rank_in_archive_order":42,"of":53,"metrics":{"Word Error Rate (WER)":"7.63"},"uses_additional_data":true}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}