{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-neural-machine-translation-with-weakly","title":"Deep Neural Machine Translation with Weakly-Recurrent Units","arxiv_id":"1805.04185","date":"2018-05-10","proceeding":null,"authors":["Mattia Antonino Di Gangi","Marcello Federico"],"abstract":"Recurrent neural networks (RNNs) have represented for years the state of the\nart in neural machine translation. Recently, new architectures have been\nproposed, which can leverage parallel computation on GPUs better than classical\nRNNs. Faster training and inference combined with different\nsequence-to-sequence modeling also lead to performance improvements. While the\nnew models completely depart from the original recurrent architecture, we\ndecided to investigate how to make RNNs more efficient. In this work, we\npropose a new recurrent NMT architecture, called Simple Recurrent NMT, built on\na class of fast and weakly-recurrent units that use layer normalization and\nmultiple attentions. Our experiments on the WMT14 English-to-German and WMT16\nEnglish-Romanian benchmarks show that our model represents a valid alternative\nto LSTMs, as it can achieve better results at a significantly lower\ncomputational cost.","url_abs":"http://arxiv.org/abs/1805.04185v1","url_pdf":"http://arxiv.org/pdf/1805.04185v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-neural-machine-translation-with-weakly","repo_url":"https://github.com/mattiadg/SR-NMT","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"nmt","task_name":"NMT"},{"task_slug":"translation","task_name":"Translation"},{"task_slug":null,"task_name":"valid"}],"methods":[{"method_slug":"layer-normalization","method_name":"Layer Normalization"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}