{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/twin-regularization-for-online-speech","title":"Twin Regularization for online speech recognition","arxiv_id":"1804.05374","date":"2018-04-15","proceeding":null,"authors":["Mirco Ravanelli","Dmitriy Serdyuk","Yoshua Bengio"],"abstract":"Online speech recognition is crucial for developing natural human-machine\ninterfaces. This modality, however, is significantly more challenging than\noff-line ASR, since real-time/low-latency constraints inevitably hinder the use\nof future information, that is known to be very helpful to perform robust\npredictions. A popular solution to mitigate this issue consists of feeding\nneural acoustic models with context windows that gather some future frames.\nThis introduces a latency which depends on the number of employed look-ahead\nfeatures. This paper explores a different approach, based on estimating the\nfuture rather than waiting for it. Our technique encourages the hidden\nrepresentations of a unidirectional recurrent network to embed some useful\ninformation about the future. Inspired by a recently proposed technique called\nTwin Networks, we add a regularization term that forces forward hidden states\nto be as close as possible to cotemporal backward ones, computed by a \"twin\"\nneural network running backwards in time. The experiments, conducted on a\nnumber of datasets, recurrent architectures, input features, and acoustic\nconditions, have shown the effectiveness of this approach. One important\nadvantage is that our method does not introduce any additional computation at\ntest time if compared to standard unidirectional recurrent networks.","url_abs":"http://arxiv.org/abs/1804.05374v2","url_pdf":"http://arxiv.org/pdf/1804.05374v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"twin-regularization-for-online-speech","repo_url":"https://github.com/mravanelli/pytorch-kaldi","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"twin-regularization-for-online-speech","repo_url":"https://github.com/MindSpore-paper-code-2/code3/tree/main/Twins","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"speech-recognition","task_name":"Speech Recognition"},{"task_slug":"speech-recognition-1","task_name":"speech-recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}