{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-better-internal-structure-of-words","title":"Learning Better Internal Structure of Words for Sequence Labeling","arxiv_id":"1810.12443","date":"2018-10-29","proceeding":"EMNLP 2018 10","authors":["Yingwei Xin","Ethan Hart","Vibhuti Mahajan","Jean-David Ruvini"],"abstract":"Character-based neural models have recently proven very useful for many NLP\ntasks. However, there is a gap of sophistication between methods for learning\nrepresentations of sentences and words. While most character models for\nlearning representations of sentences are deep and complex, models for learning\nrepresentations of words are shallow and simple. Also, in spite of considerable\nresearch on learning character embeddings, it is still not clear which kind of\narchitecture is the best for capturing character-to-word representations. To\naddress these questions, we first investigate the gaps between methods for\nlearning word and sentence representations. We conduct detailed experiments and\ncomparisons of different state-of-the-art convolutional models, and also\ninvestigate the advantages and disadvantages of their constituents.\nFurthermore, we propose IntNet, a funnel-shaped wide convolutional neural\narchitecture with no down-sampling for learning representations of the internal\nstructure of words by composing their characters from limited, supervised\ntraining corpora. We evaluate our proposed model on six sequence labeling\ndatasets, including named entity recognition, part-of-speech tagging, and\nsyntactic chunking. Our in-depth analysis shows that IntNet significantly\noutperforms other character embedding models and obtains new state-of-the-art\nperformance without relying on any external knowledge or resources.","url_abs":"http://arxiv.org/abs/1810.12443v1","url_pdf":"http://arxiv.org/pdf/1810.12443v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"chunking","task_name":"Chunking"},{"task_slug":"named-entity-recognition-1","task_name":"Named Entity Recognition"},{"task_slug":"named-entity-recognition-ner","task_name":"Named Entity Recognition (NER)"},{"task_slug":"part-of-speech-tagging","task_name":"Part-Of-Speech Tagging"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"named-entity-recognition","task_name":"named-entity-recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/chunking-on-penn-treebank","task":"Chunking","dataset":"Penn Treebank","model":"IntNet + BiLSTM-CRF","rank_in_archive_order":5,"of":8,"metrics":{"F1 score":"95.29"},"uses_additional_data":false},{"leaderboard":"/sota/named-entity-recognition-ner-on-conll-2003","task":"Named Entity Recognition (NER)","dataset":"CoNLL 2003 (English)","model":"IntNet + BiLSTM-CRF","rank_in_archive_order":56,"of":73,"metrics":{"F1":"91.64"},"uses_additional_data":false},{"leaderboard":"/sota/part-of-speech-tagging-on-penn-treebank","task":"Part-Of-Speech Tagging","dataset":"Penn Treebank","model":"IntNet + BiLSTM-CRF","rank_in_archive_order":9,"of":20,"metrics":{"Accuracy":"97.58"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1810.12443","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}