{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-simple-language-model-for-task-oriented","title":"A Simple Language Model for Task-Oriented Dialogue","arxiv_id":"2005.00796","date":"2020-05-02","proceeding":"NeurIPS 2020 12","authors":["Ehsan Hosseini-Asl","Bryan McCann","Chien-Sheng Wu","Semih Yavuz","Richard Socher"],"abstract":"Task-oriented dialogue is often decomposed into three tasks: understanding user input, deciding actions, and generating a response. While such decomposition might suggest a dedicated model for each sub-task, we find a simple, unified approach leads to state-of-the-art performance on the MultiWOZ dataset. SimpleTOD is a simple approach to task-oriented dialogue that uses a single, causal language model trained on all sub-tasks recast as a single sequence prediction problem. This allows SimpleTOD to fully leverage transfer learning from pre-trained, open domain, causal language models such as GPT-2. SimpleTOD improves over the prior state-of-the-art in joint goal accuracy for dialogue state tracking, and our analysis reveals robustness to noisy annotations in this setting. SimpleTOD also improves the main metrics used to evaluate action decisions and response generation in an end-to-end setting: inform rate by 8.1 points, success rate by 9.7 points, and combined score by 7.2 points.","url_abs":"https://arxiv.org/abs/2005.00796v4","url_pdf":"https://arxiv.org/pdf/2005.00796v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-simple-language-model-for-task-oriented","repo_url":"https://github.com/salesforce/simpletod","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"tasks":[{"task_slug":"dialogue-state-tracking","task_name":"Dialogue State Tracking"},{"task_slug":"end-to-end-dialogue-modelling","task_name":"End-To-End Dialogue Modelling"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"multi-domain-dialogue-state-tracking","task_name":"Multi-domain Dialogue State Tracking"},{"task_slug":"response-generation","task_name":"Response Generation"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"},{"task_slug":"model","task_name":"model"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"cosine-annealing","method_name":"Cosine Annealing"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"discriminative-fine-tuning","method_name":"Discriminative Fine-Tuning"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt-2","method_name":"GPT-2"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-cosine-annealing","method_name":"Linear Warmup With Cosine Annealing"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/end-to-end-dialogue-modelling-on-multiwoz-2-0","task":"End-To-End Dialogue Modelling","dataset":"MULTIWOZ 2.0","model":"SimpleTOD","rank_in_archive_order":5,"of":6,"metrics":{"BLEU":"15.0","MultiWOZ (Inform)":"84.4","MultiWOZ (Success)":"70.1"},"uses_additional_data":false},{"leaderboard":"/sota/end-to-end-dialogue-modelling-on-multiwoz-2-1","task":"End-To-End Dialogue Modelling","dataset":"MULTIWOZ 2.1","model":"SimpleTOD","rank_in_archive_order":4,"of":4,"metrics":{"BLEU":"15.2","MultiWOZ (Inform)":"85.0","MultiWOZ (Success)":"70.5"},"uses_additional_data":false},{"leaderboard":"/sota/response-generation-on-mmconv","task":"Response Generation","dataset":"MMConv","model":"SimpleTOD","rank_in_archive_order":2,"of":2,"metrics":{"BLEU":"20.3","Comb.":"32.2","Inform":"14.6","Success":"9.2"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2005.00796","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2005.00796"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/salesforce/simpletod","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"summary":{"ran_violates":1,"ran_draft_wrong":1,"ran":1,"unverified":11},"by_repo_kind":{"official":{"samples":14,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1b465b9239cb631c","entry":"is_ascii","repo":"salesforce/simpletod","repo_kind":"official","path":"preprocess_multiwoz.py","file_url":"https://github.com/salesforce/simpletod/blob/HEAD/preprocess_multiwoz.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"1b465b9239cb631c"}},{"code_sha256_prefix":"d30d6c3098df2c42","entry":"prune_linear_layer","repo":"salesforce/simpletod","repo_kind":"official","path":"models/modeling_utils.py","file_url":"https://github.com/salesforce/simpletod/blob/HEAD/models/modeling_utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"d30d6c3098df2c42"}},{"code_sha256_prefix":"e827d94caab33f59","entry":"url_to_filename","repo":"salesforce/simpletod","repo_kind":"official","path":"models/file_utils.py","file_url":"https://github.com/salesforce/simpletod/blob/HEAD/models/file_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"e827d94caab33f59"}},{"code_sha256_prefix":"d4f3546f2ecff417","entry":"create_position_ids_from_input_ids","repo":"salesforce/simpletod","repo_kind":"official","path":"models/modeling_utils.py","file_url":"https://github.com/salesforce/simpletod/blob/HEAD/models/modeling_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"d4f3546f2ecff417"}},{"code_sha256_prefix":"9db2246dbf2fe48c","entry":"fixDelex","repo":"salesforce/simpletod","repo_kind":"official","path":"preprocess_multiwoz.py","file_url":"https://github.com/salesforce/simpletod/blob/HEAD/preprocess_multiwoz.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"9db2246dbf2fe48c"}},{"code_sha256_prefix":"6593f647e65bca52","entry":"get_belief","repo":"salesforce/simpletod","repo_kind":"official","path":"utils/simpletod.py","file_url":"https://github.com/salesforce/simpletod/blob/HEAD/utils/simpletod.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"6593f647e65bca52"}},{"code_sha256_prefix":"d273f88c13f089a1","entry":"get_belief_dbsearch","repo":"salesforce/simpletod","repo_kind":"official","path":"utils/simpletod.py","file_url":"https://github.com/salesforce/simpletod/blob/HEAD/utils/simpletod.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"d273f88c13f089a1"}},{"code_sha256_prefix":"817ada1e305378b6","entry":"get_belief_openaigpt","repo":"salesforce/simpletod","repo_kind":"official","path":"utils/simpletod.py","file_url":"https://github.com/salesforce/simpletod/blob/HEAD/utils/simpletod.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"817ada1e305378b6"}},{"code_sha256_prefix":"b2be6c932ce41084","entry":"hf_bucket_url","repo":"salesforce/simpletod","repo_kind":"official","path":"models/file_utils.py","file_url":"https://github.com/salesforce/simpletod/blob/HEAD/models/file_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"b2be6c932ce41084"}},{"code_sha256_prefix":"4381d1f588a7f42a","entry":"is_remote_url","repo":"salesforce/simpletod","repo_kind":"official","path":"models/file_utils.py","file_url":"https://github.com/salesforce/simpletod/blob/HEAD/models/file_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"4381d1f588a7f42a"}},{"code_sha256_prefix":"c4bac98236c1fc2c","entry":"load_tf_weights_in_gpt2","repo":"salesforce/simpletod","repo_kind":"official","path":"models/modeling_gpt2.py","file_url":"https://github.com/salesforce/simpletod/blob/HEAD/models/modeling_gpt2.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"c4bac98236c1fc2c"}},{"code_sha256_prefix":"9b5c8cad10804374","entry":"postprocess_gpt2","repo":"salesforce/simpletod","repo_kind":"official","path":"evaluate_multiwoz.py","file_url":"https://github.com/salesforce/simpletod/blob/HEAD/evaluate_multiwoz.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"9b5c8cad10804374"}},{"code_sha256_prefix":"489b09c8fef69f12","entry":"remove_model_mismatch_and_db_data","repo":"salesforce/simpletod","repo_kind":"official","path":"evaluate_multiwoz.py","file_url":"https://github.com/salesforce/simpletod/blob/HEAD/evaluate_multiwoz.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"489b09c8fef69f12"}},{"code_sha256_prefix":"8e873fc150477bd1","entry":"top_k_top_p_filtering","repo":"salesforce/simpletod","repo_kind":"official","path":"models/modeling_utils.py","file_url":"https://github.com/salesforce/simpletod/blob/HEAD/models/modeling_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"8e873fc150477bd1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}