{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/chai-a-chatbot-ai-for-task-oriented-dialogue","title":"CHAI: A CHatbot AI for Task-Oriented Dialogue with Offline Reinforcement Learning","arxiv_id":"2204.08426","date":"2022-04-18","proceeding":"NAACL 2022 7","authors":["Siddharth Verma","Justin Fu","Mengjiao Yang","Sergey Levine"],"abstract":"Conventionally, generation of natural language for dialogue agents may be viewed as a statistical learning problem: determine the patterns in human-provided data and generate appropriate responses with similar statistical properties. However, dialogue can also be regarded as a goal directed process, where speakers attempt to accomplish a specific task. Reinforcement learning (RL) algorithms are designed specifically for solving such goal-directed problems, but the most direct way to apply RL -- through trial-and-error learning in human conversations, -- is costly. In this paper, we study how offline reinforcement learning can instead be used to train dialogue agents entirely using static datasets collected from human speakers. Our experiments show that recently developed offline RL methods can be combined with language models to yield realistic dialogue agents that better accomplish task goals.","url_abs":"https://arxiv.org/abs/2204.08426v1","url_pdf":"https://arxiv.org/pdf/2204.08426v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"chai-a-chatbot-ai-for-task-oriented-dialogue","repo_url":"https://github.com/siddharthverma314/chai-naacl-2022","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"chai-a-chatbot-ai-for-task-oriented-dialogue","repo_url":"https://github.com/yifeizhou02/archer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"chatbot","task_name":"Chatbot"},{"task_slug":"offline-rl","task_name":"Offline RL"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2204.08426","atlas_url":"https://app.syntology.ai/?focus=2204.08426","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2204.08426"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yifeizhou02/archer","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/siddharthverma314/chai-naacl-2022","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":4,"unverified":3},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1},"listed":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0b2ec4d5f7ded4de","entry":"generate_uuid","repo":"siddharthverma314/chai-naacl-2022","repo_kind":"official","path":"cocoa/core/util.py","file_url":"https://github.com/siddharthverma314/chai-naacl-2022/blob/HEAD/cocoa/core/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0b2ec4d5f7ded4de"}},{"code_sha256_prefix":"79624785a930cf93","entry":"is_entity","repo":"siddharthverma314/chai-naacl-2022","repo_kind":"official","path":"cocoa/core/entity.py","file_url":"https://github.com/siddharthverma314/chai-naacl-2022/blob/HEAD/cocoa/core/entity.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"79624785a930cf93"}},{"code_sha256_prefix":"051b6d3efcdab50c","entry":"random_multinomial","repo":"siddharthverma314/chai-naacl-2022","repo_kind":"official","path":"cocoa/core/util.py","file_url":"https://github.com/siddharthverma314/chai-naacl-2022/blob/HEAD/cocoa/core/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"051b6d3efcdab50c"}},{"code_sha256_prefix":"51b941b0a4c27d3d","entry":"read_json","repo":"siddharthverma314/chai-naacl-2022","repo_kind":"official","path":"cocoa/core/util.py","file_url":"https://github.com/siddharthverma314/chai-naacl-2022/blob/HEAD/cocoa/core/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"51b941b0a4c27d3d"}},{"code_sha256_prefix":"c413b3d673c5bb86","entry":"CHAIAgent","repo":"yifeizhou02/archer","repo_kind":"listed","path":"archer/models/chai_agent.py","file_url":"https://github.com/yifeizhou02/archer/blob/HEAD/archer/models/chai_agent.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c413b3d673c5bb86"}},{"code_sha256_prefix":"3c707b3a78f390d0","entry":"DoubleCritic","repo":"yifeizhou02/archer","repo_kind":"listed","path":"archer/models/chai_agent.py","file_url":"https://github.com/yifeizhou02/archer/blob/HEAD/archer/models/chai_agent.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3c707b3a78f390d0"}},{"code_sha256_prefix":"702b27974999a98a","entry":"tokenize","repo":"siddharthverma314/chai-naacl-2022","repo_kind":"official","path":"cocoa/core/tokenizer.py","file_url":"https://github.com/siddharthverma314/chai-naacl-2022/blob/HEAD/cocoa/core/tokenizer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"702b27974999a98a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}