{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/toward-diverse-text-generation-with-inverse","title":"Toward Diverse Text Generation with Inverse Reinforcement Learning","arxiv_id":"1804.11258","date":"2018-04-30","proceeding":null,"authors":["Zhan Shi","Xinchi Chen","Xipeng Qiu","Xuanjing Huang"],"abstract":"Text generation is a crucial task in NLP. Recently, several adversarial\ngenerative models have been proposed to improve the exposure bias problem in\ntext generation. Though these models gain great success, they still suffer from\nthe problems of reward sparsity and mode collapse. In order to address these\ntwo problems, in this paper, we employ inverse reinforcement learning (IRL) for\ntext generation. Specifically, the IRL framework learns a reward function on\ntraining data, and then an optimal policy to maximum the expected total reward.\nSimilar to the adversarial models, the reward and policy function in IRL are\noptimized alternately. Our method has two advantages: (1) the reward function\ncan produce more dense reward signals. (2) the generation policy, trained by\n\"entropy regularized\" policy gradient, encourages to generate more diversified\ntexts. Experiment results demonstrate that our proposed method can generate\nhigher quality texts than the previous methods.","url_abs":"http://arxiv.org/abs/1804.11258v3","url_pdf":"http://arxiv.org/pdf/1804.11258v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"toward-diverse-text-generation-with-inverse","repo_url":"https://github.com/FudanNLP/Irl_gen","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"toward-diverse-text-generation-with-inverse","repo_url":"https://github.com/ronakdm/irl-text-generation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"toward-diverse-text-generation-with-inverse","repo_url":"https://github.com/tanyuqian/progressive-generation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"text-generation","task_name":"Text Generation"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1804.11258","atlas_url":"https://app.syntology.ai/?focus=1804.11258","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}