{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gated-word-character-recurrent-language-model","title":"Gated Word-Character Recurrent Language Model","arxiv_id":"1606.01700","date":"2016-06-06","proceeding":"EMNLP 2016 11","authors":["Yasumasa Miyamoto","Kyunghyun Cho"],"abstract":"We introduce a recurrent neural network language model (RNN-LM) with long\nshort-term memory (LSTM) units that utilizes both character-level and\nword-level inputs. Our model has a gate that adaptively finds the optimal\nmixture of the character-level and word-level inputs. The gate creates the\nfinal vector representation of a word by combining two distinct representations\nof the word. The character-level inputs are converted into vector\nrepresentations of words using a bidirectional LSTM. The word-level inputs are\nprojected into another high-dimensional space by a word lookup table. The final\nvector representations of words are used in the LSTM language model which\npredicts the next word given all the preceding words. Our model with the gating\nmechanism effectively utilizes the character-level inputs for rare and\nout-of-vocabulary words and outperforms word-level language models on several\nEnglish corpora.","url_abs":"http://arxiv.org/abs/1606.01700v2","url_pdf":"http://arxiv.org/pdf/1606.01700v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"gated-word-character-recurrent-language-model","repo_url":"https://github.com/nyu-dl/gated_word_char_rlm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"gated-word-character-recurrent-language-model","repo_url":"https://github.com/arshadshk/GatedWord-Character_Recurrent_Language_Model","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"gated-word-character-recurrent-language-model","repo_url":"https://github.com/MindCode-4/code-12/tree/main/gated-word-character-recurrent-language-model","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"gated-word-character-recurrent-language-model","repo_url":"https://github.com/MindCode-4/code-7/tree/main/gated-word-character-recurrent-language-model","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"gated-word-character-recurrent-language-model","repo_url":"https://github.com/MindSpore-scientific-2/code-12/tree/main/gated-word-character-recurrent-language-model","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"model","task_name":"model"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}