{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/glm-dialog-noise-tolerant-pre-training-for","title":"GLM-Dialog: Noise-tolerant Pre-training for Knowledge-grounded Dialogue Generation","arxiv_id":"2302.14401","date":"2023-02-28","proceeding":null,"authors":["Jing Zhang","Xiaokang Zhang","Daniel Zhang-li","Jifan Yu","Zijun Yao","Zeyao Ma","Yiqi Xu","Haohua Wang","Xiaohan Zhang","Nianyi Lin","Sunrui Lu","Juanzi Li","Jie Tang"],"abstract":"We present GLM-Dialog, a large-scale language model (LLM) with 10B parameters capable of knowledge-grounded conversation in Chinese using a search engine to access the Internet knowledge. GLM-Dialog offers a series of applicable techniques for exploiting various external knowledge including both helpful and noisy knowledge, enabling the creation of robust knowledge-grounded dialogue LLMs with limited proper datasets. To evaluate the GLM-Dialog more fairly, we also propose a novel evaluation method to allow humans to converse with multiple deployed bots simultaneously and compare their performance implicitly instead of explicitly rating using multidimensional metrics.Comprehensive evaluations from automatic to human perspective demonstrate the advantages of GLM-Dialog comparing with existing open source Chinese dialogue models. We release both the model checkpoint and source code, and also deploy it as a WeChat application to interact with users. We offer our evaluation platform online in an effort to prompt the development of open source models and reliable dialogue evaluation systems. The additional easy-to-use toolkit that consists of short text entity linking, query generation, and helpful knowledge classification is also released to enable diverse applications. All the source code is available on Github.","url_abs":"https://arxiv.org/abs/2302.14401v1","url_pdf":"https://arxiv.org/pdf/2302.14401v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"glm-dialog-noise-tolerant-pre-training-for","repo_url":"https://github.com/ruckbreasoning/glm-dialog","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"dialogue-evaluation","task_name":"Dialogue Evaluation"},{"task_slug":"dialogue-generation","task_name":"Dialogue Generation"},{"task_slug":"entity-linking","task_name":"Entity Linking"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2302.14401","atlas_url":"https://app.syntology.ai/?focus=2302.14401","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}