{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/vihealthbert-pre-trained-language-models-for","title":"ViHealthBERT: Pre-trained Language Models for Vietnamese in Health Text Mining","arxiv_id":null,"date":"2022-06-01","proceeding":"LREC 2022 6","authors":["Minh","Nguyen  and  Tran","Vu Hoang  and  Hoang","Vu  and  Ta","Huy Duc  and  Bui","Trung Huu  and  Truong","Steven Quoc Hung"],"abstract":"Pre-trained language models have become crucial to achieving competitive results across many Natural Language Processing (NLP) problems. For monolingual pre-trained models in low-resource languages, the quantity has been significantly increased. However, most of them relate to the general domain, and there are limited strong baseline language models for domain-specific. We introduce ViHealthBERT, the first domain-specific pre-trained language model for Vietnamese healthcare. The performance of our model shows strong results while outperforming the general domain language models in all health-related datasets. Moreover, we also present Vietnamese datasets for the healthcare domain for two tasks are Acronym Disambiguation (AD) and Frequently Asked Questions (FAQ) Summarization. We release our ViHealthBERT to facilitate future research and downstream application for Vietnamese NLP in domain-specific.","url_abs":"http://www.lrec-conf.org/proceedings/lrec2022/pdf/2022.lrec-1.35.pdf","url_pdf":"http://www.lrec-conf.org/proceedings/lrec2022/pdf/2022.lrec-1.35.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"vihealthbert-pre-trained-language-models-for","repo_url":"https://github.com/demdecuong/vihealthbert","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"medical-named-entity-recognition","task_name":"Medical Named Entity Recognition"},{"task_slug":"named-entity-recognition-in-vietnamese","task_name":"Named Entity Recognition In Vietnamese"},{"task_slug":"text-summarization","task_name":"Text Summarization"},{"task_slug":"vietnamese-datasets","task_name":"Vietnamese Datasets"},{"task_slug":"word-sense-disambiguation","task_name":"Word Sense Disambiguation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/named-entity-recognition-in-vietnamese-on-1","task":"Named Entity Recognition In Vietnamese","dataset":"PhoNER COVID19","model":"ViHealthBERT","rank_in_archive_order":3,"of":3,"metrics":{"F1 (%)":"96.7"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}