{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/recurrent-neural-networks-with-pre-trained","title":"Recurrent Neural Networks with Pre-trained Language Model Embedding for Slot Filling Task","arxiv_id":"1812.05199","date":"2018-12-12","proceeding":null,"authors":["Liang Qiu","Yuanyi Ding","Lei He"],"abstract":"In recent years, Recurrent Neural Networks (RNNs) based models have been\napplied to the Slot Filling problem of Spoken Language Understanding and\nachieved the state-of-the-art performances. In this paper, we investigate the\neffect of incorporating pre-trained language models into RNN based Slot Filling\nmodels. Our evaluation on the Airline Travel Information System (ATIS) data\ncorpus shows that we can significantly reduce the size of labeled training data\nand achieve the same level of Slot Filling performance by incorporating extra\nword embedding and language model embedding layers pre-trained on unlabeled\ncorpora.","url_abs":"http://arxiv.org/abs/1812.05199v2","url_pdf":"http://arxiv.org/pdf/1812.05199v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"recurrent-neural-networks-with-pre-trained","repo_url":"https://github.com/Liang-Qiu/is13","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"slot-filling","task_name":"Slot Filling"},{"task_slug":"spoken-language-understanding","task_name":"Spoken Language Understanding"},{"task_slug":"slot-filling-1","task_name":"slot-filling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}