{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/recurrent-neural-networks-with-specialized","title":"Recurrent neural networks with specialized word embeddings for health-domain named-entity recognition","arxiv_id":"1706.09569","date":"2017-06-29","proceeding":null,"authors":["Inigo Jauregi Unanue","Ehsan Zare Borzeshi","Massimo Piccardi"],"abstract":"Background. Previous state-of-the-art systems on Drug Name Recognition (DNR)\nand Clinical Concept Extraction (CCE) have focused on a combination of text\n\"feature engineering\" and conventional machine learning algorithms such as\nconditional random fields and support vector machines. However, developing good\nfeatures is inherently heavily time-consuming. Conversely, more modern machine\nlearning approaches such as recurrent neural networks (RNNs) have proved\ncapable of automatically learning effective features from either random\nassignments or automated word \"embeddings\". Objectives. (i) To create a highly\naccurate DNR and CCE system that avoids conventional, time-consuming feature\nengineering. (ii) To create richer, more specialized word embeddings by using\nhealth domain datasets such as MIMIC-III. (iii) To evaluate our systems over\nthree contemporary datasets. Methods. Two deep learning methods, namely the\nBidirectional LSTM and the Bidirectional LSTM-CRF, are evaluated. A CRF model\nis set as the baseline to compare the deep learning systems to a traditional\nmachine learning approach. The same features are used for all the models.\nResults. We have obtained the best results with the Bidirectional LSTM-CRF\nmodel, which has outperformed all previously proposed systems. The specialized\nembeddings have helped to cover unusual words in DDI-DrugBank and DDI-MedLine,\nbut not in the 2010 i2b2/VA IRB Revision dataset. Conclusion. We present a\nstate-of-the-art system for DNR and CCE. Automated word embeddings has allowed\nus to avoid costly feature engineering and achieve higher accuracy.\nNevertheless, the embeddings need to be retrained over datasets that are\nadequate for the domain, in order to adequately cover the domain-specific\nvocabulary.","url_abs":"http://arxiv.org/abs/1706.09569v2","url_pdf":"http://arxiv.org/pdf/1706.09569v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"recurrent-neural-networks-with-specialized","repo_url":"https://github.com/ijauregiCMCRC/healthNER","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"machine-learning","task_name":"BIG-bench Machine Learning"},{"task_slug":"clinical-concept-extraction","task_name":"Clinical Concept Extraction"},{"task_slug":"feature-engineering","task_name":"Feature Engineering"},{"task_slug":"named-entity-recognition-1","task_name":"Named Entity Recognition"},{"task_slug":"named-entity-recognition-ner","task_name":"Named Entity Recognition (NER)"},{"task_slug":"word-embeddings","task_name":"Word Embeddings"},{"task_slug":"named-entity-recognition","task_name":"named-entity-recognition"}],"methods":[{"method_slug":"crf","method_name":"CRF"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}