{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dual-long-short-term-memory-networks-for-sub","title":"Dual Long Short-Term Memory Networks for Sub-Character Representation Learning","arxiv_id":"1712.08841","date":"2017-12-23","proceeding":null,"authors":["Han He","Lei Wu","Xiaokun Yang","Hua Yan","Zhimin Gao","Yi Feng","George Townsend"],"abstract":"Characters have commonly been regarded as the minimal processing unit in\nNatural Language Processing (NLP). But many non-latin languages have\nhieroglyphic writing systems, involving a big alphabet with thousands or\nmillions of characters. Each character is composed of even smaller parts, which\nare often ignored by the previous work. In this paper, we propose a novel\narchitecture employing two stacked Long Short-Term Memory Networks (LSTMs) to\nlearn sub-character level representation and capture deeper level of semantic\nmeanings. To build a concrete study and substantiate the efficiency of our\nneural architecture, we take Chinese Word Segmentation as a research case\nexample. Among those languages, Chinese is a typical case, for which every\ncharacter contains several components called radicals. Our networks employ a\nshared radical level embedding to solve both Simplified and Traditional Chinese\nWord Segmentation, without extra Traditional to Simplified Chinese conversion,\nin such a highly end-to-end way the word segmentation can be significantly\nsimplified compared to the previous work. Radical level embeddings can also\ncapture deeper semantic meaning below character level and improve the system\nperformance of learning. By tying radical and character embeddings together,\nthe parameter count is reduced whereas semantic knowledge is shared and\ntransferred between two levels, boosting the performance largely. On 3 out of 4\nBakeoff 2005 datasets, our method surpassed state-of-the-art results by up to\n0.4%. Our results are reproducible, source codes and corpora are available on\nGitHub.","url_abs":"http://arxiv.org/abs/1712.08841v2","url_pdf":"http://arxiv.org/pdf/1712.08841v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dual-long-short-term-memory-networks-for-sub","repo_url":"https://github.com/hankcs/sub-character-cws","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"chinese-word-segmentation","task_name":"Chinese Word Segmentation"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"segmentation","task_name":"Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}