{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/attending-to-characters-in-neural-sequence","title":"Attending to Characters in Neural Sequence Labeling Models","arxiv_id":"1611.04361","date":"2016-11-14","proceeding":"COLING 2016 12","authors":["Marek Rei","Gamal K. O. Crichton","Sampo Pyysalo"],"abstract":"Sequence labeling architectures use word embeddings for capturing similarity,\nbut suffer when handling previously unseen or rare words. We investigate\ncharacter-level extensions to such models and propose a novel architecture for\ncombining alternative word representations. By using an attention mechanism,\nthe model is able to dynamically decide how much information to use from a\nword- or character-level component. We evaluated different architectures on a\nrange of sequence labeling datasets, and character-level extensions were found\nto improve performance on every benchmark. In addition, the proposed\nattention-based architecture delivered the best results even with a smaller\nnumber of trainable parameters.","url_abs":"http://arxiv.org/abs/1611.04361v1","url_pdf":"http://arxiv.org/pdf/1611.04361v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"chunking","task_name":"Chunking"},{"task_slug":"grammatical-error-detection","task_name":"Grammatical Error Detection"},{"task_slug":"named-entity-recognition-ner","task_name":"Named Entity Recognition (NER)"},{"task_slug":"part-of-speech-tagging","task_name":"Part-Of-Speech Tagging"},{"task_slug":"word-embeddings","task_name":"Word Embeddings"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/grammatical-error-detection-on-fce","task":"Grammatical Error Detection","dataset":"FCE","model":"Bi-LSTM + charattn","rank_in_archive_order":7,"of":8,"metrics":{"F0.5":"41.88"},"uses_additional_data":false},{"leaderboard":"/sota/part-of-speech-tagging-on-penn-treebank","task":"Part-Of-Speech Tagging","dataset":"Penn Treebank","model":"Bi-LSTM + charattn","rank_in_archive_order":18,"of":20,"metrics":{"Accuracy":"97.27"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1611.04361","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}