{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/character-aware-neural-language-models","title":"Character-Aware Neural Language Models","arxiv_id":"1508.06615","date":"2015-08-26","proceeding":null,"authors":["Yoon Kim","Yacine Jernite","David Sontag","Alexander M. Rush"],"abstract":"We describe a simple neural language model that relies only on\ncharacter-level inputs. Predictions are still made at the word-level. Our model\nemploys a convolutional neural network (CNN) and a highway network over\ncharacters, whose output is given to a long short-term memory (LSTM) recurrent\nneural network language model (RNN-LM). On the English Penn Treebank the model\nis on par with the existing state-of-the-art despite having 60% fewer\nparameters. On languages with rich morphology (Arabic, Czech, French, German,\nSpanish, Russian), the model outperforms word-level/morpheme-level LSTM\nbaselines, again with fewer parameters. The results suggest that on many\nlanguages, character inputs are sufficient for language modeling. Analysis of\nword representations obtained from the character composition part of the model\nreveals that the model is able to encode, from characters only, both semantic\nand orthographic information.","url_abs":"http://arxiv.org/abs/1508.06615v4","url_pdf":"http://arxiv.org/pdf/1508.06615v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"character-aware-neural-language-models","repo_url":"https://github.com/yoonkim/lstm-char-cnn","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"torch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"character-aware-neural-language-models","repo_url":"https://github.com/NLPLearn/QANet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"character-aware-neural-language-models","repo_url":"https://github.com/SNUDerek/multiLSTM","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"character-aware-neural-language-models","repo_url":"https://github.com/SeonbeomKim/TensorFlow-lstm-char-cnn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"character-aware-neural-language-models","repo_url":"https://github.com/chanedwin/kaggletoxic","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"character-aware-neural-language-models","repo_url":"https://github.com/davidsvaughn/dts-tf","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"character-aware-neural-language-models","repo_url":"https://github.com/hansungj/CharCNN_Pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"character-aware-neural-language-models","repo_url":"https://github.com/jarfo/kchar","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"torch","reach":null},{"paper_slug":"character-aware-neural-language-models","repo_url":"https://github.com/lmtoan/nlp-cs224n","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"character-aware-neural-language-models","repo_url":"https://github.com/localminimum/qanet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"character-aware-neural-language-models","repo_url":"https://github.com/mhjabreel/CharCnn_Keras","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"character-aware-neural-language-models","repo_url":"https://github.com/seharanul17/RNN-LM","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"character-aware-neural-language-models","repo_url":"https://github.com/stikbuf/Language_Modeling","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"character-aware-neural-language-models","repo_url":"https://github.com/sucheta19/Text-Classification-Using-CNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[{"method_slug":"highway-layer","method_name":"Highway Layer"},{"method_slug":"highway-network","method_name":"Highway Network"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1508.06615","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1508.06615"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SeonbeomKim/TensorFlow-lstm-char-cnn","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yoonkim/lstm-char-cnn","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/localminimum/qanet","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/seharanul17/RNN-LM","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sucheta19/Text-Classification-Using-CNN","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SNUDerek/multiLSTM","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mhjabreel/CharCnn_Keras","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hansungj/CharCNN_Pytorch","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/chanedwin/kaggletoxic","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/NLPLearn/QANet","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lmtoan/nlp-cs224n","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/stikbuf/Language_Modeling","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/davidsvaughn/dts-tf","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jarfo/kchar","reach":null}],"summary":{"ran_violates":1,"ran_honours":1,"ran_draft_wrong":2,"unverified":7},"by_repo_kind":{"listed":{"samples":11,"ran":4,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"f6c275d6a18330a9","entry":"exact_match_score","repo":"NLPLearn/QANet","repo_kind":"listed","path":"evaluate-v1.1.py","file_url":"https://github.com/NLPLearn/QANet/blob/HEAD/evaluate-v1.1.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f6c275d6a18330a9"}},{"code_sha256_prefix":"2112c433b9c6d343","entry":"f1_score","repo":"NLPLearn/QANet","repo_kind":"listed","path":"evaluate-v1.1.py","file_url":"https://github.com/NLPLearn/QANet/blob/HEAD/evaluate-v1.1.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2112c433b9c6d343"}},{"code_sha256_prefix":"c6a80c065d2e4851","entry":"normalize_answer","repo":"NLPLearn/QANet","repo_kind":"listed","path":"evaluate-v1.1.py","file_url":"https://github.com/NLPLearn/QANet/blob/HEAD/evaluate-v1.1.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c6a80c065d2e4851"}},{"code_sha256_prefix":"f45d0a8d1a90a693","entry":"vocab_unpack","repo":"jarfo/kchar","repo_kind":"listed","path":"evaluate.py","file_url":"https://github.com/jarfo/kchar/blob/HEAD/evaluate.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f45d0a8d1a90a693"}},{"code_sha256_prefix":"23587f1e97b0e35d","entry":"convert_idx","repo":"NLPLearn/QANet","repo_kind":"listed","path":"prepro.py","file_url":"https://github.com/NLPLearn/QANet/blob/HEAD/prepro.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"23587f1e97b0e35d"}},{"code_sha256_prefix":"062d8efac782ef22","entry":"get_batch_dataset","repo":"NLPLearn/QANet","repo_kind":"listed","path":"util.py","file_url":"https://github.com/NLPLearn/QANet/blob/HEAD/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"062d8efac782ef22"}},{"code_sha256_prefix":"1ea08aaae7fd9f80","entry":"get_dataset","repo":"NLPLearn/QANet","repo_kind":"listed","path":"util.py","file_url":"https://github.com/NLPLearn/QANet/blob/HEAD/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1ea08aaae7fd9f80"}},{"code_sha256_prefix":"a89fcc1809e712da","entry":"get_record_parser","repo":"NLPLearn/QANet","repo_kind":"listed","path":"util.py","file_url":"https://github.com/NLPLearn/QANet/blob/HEAD/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a89fcc1809e712da"}},{"code_sha256_prefix":"f9afe5c09ca7b577","entry":"glu","repo":"NLPLearn/QANet","repo_kind":"listed","path":"layers.py","file_url":"https://github.com/NLPLearn/QANet/blob/HEAD/layers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f9afe5c09ca7b577"}},{"code_sha256_prefix":"df5f1c26bcb94d35","entry":"layer_norm_compute_python","repo":"NLPLearn/QANet","repo_kind":"listed","path":"layers.py","file_url":"https://github.com/NLPLearn/QANet/blob/HEAD/layers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"df5f1c26bcb94d35"}},{"code_sha256_prefix":"15b2eb813c7ac34f","entry":"noam_norm","repo":"NLPLearn/QANet","repo_kind":"listed","path":"layers.py","file_url":"https://github.com/NLPLearn/QANet/blob/HEAD/layers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"15b2eb813c7ac34f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}