{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/an-efficient-transformer-based-model-for","title":"An Efficient Transformer-Based Model for Vietnamese Punctuation Prediction","arxiv_id":null,"date":"2021-07-19","proceeding":"IEA/AIE 2021 7","authors":["Hieu Tran","Cuong V. Dinh","Quang Pham","Binh T. Nguyen"],"abstract":"In both formal and informal texts, missing punctuation marks make the texts confusing and challenging to read. This paper aims to conduct exhaustive experiments to investigate the benefits of the pre-trained Transformer-based models on two Vietnamese punctuation datasets. The experimental results show our models can achieve encouraging results, and adding Bi-LSTM or/and CRF layers on top of the proposed models can also boost model performance. Finally, our best model can significantly bypass state-of-the-art approaches on both the novel and news datasets for the Vietnamese language. It can gain the corresponding performance up to 21.45%  and 18.27%  in the overall F1-scores.","url_abs":"https://link.springer.com/chapter/10.1007/978-3-030-79463-7_5","url_pdf":"https://link.springer.com/chapter/10.1007/978-3-030-79463-7_5","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"an-efficient-transformer-based-model-for","repo_url":"https://github.com/heraclex12/vietpunc","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"jax","reach":null}],"tasks":[{"task_slug":"prediction","task_name":"Prediction"}],"methods":[{"method_slug":"crf","method_name":"CRF"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}