Open-Domain Dialog

Benchmarks archive 2025-07-28

1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

13 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

1 subtask in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 32 papers with code (60 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation
  16 Apr 2021 3 repositories listed
  ProphetNet is a pre-training based natural language generation method which shows powerful performance on English text summarization and question generation tasks.
• KILT: a Benchmark for Knowledge Intensive Language Tasks
  4 Sep 2020 3 repositories listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  We test both task-specific and general baselines, evaluating downstream performance in addition to the ability of the models to provide provenance.
• Hurdles to Progress in Long-form Question Answering
  10 Mar 2021 2 repositories listed
  The task of long-form question answering (LFQA) involves retrieving documents relevant to a given question and using them to generate a paragraph-length answer.
• Dialogue Response Ranking Training with Large-Scale Human Feedback Data
  15 Sep 2020 2 repositories listed Syntology ran 1 of 6 samples · 5 unverified
  Particularly, our ranker outperforms the conventional dialog perplexity baseline with a large margin on predicting Reddit feedback.
• Unsupervised Evaluation of Interactive Dialog with DialoGPT
  23 Jun 2020 2 repositories listed Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
  It is important to define meaningful and interpretable automatic evaluation metrics for open-domain dialog research.
• Investigating Evaluation of Open-Domain Dialogue Systems With Human Generated Multiple References
  24 Jul 2019 2 repositories listed Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
  The aim of this paper is to mitigate the shortcomings of automatic evaluation of open-domain dialog systems through multi-reference evaluation.
• Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems
  21 Jun 2019 2 repositories listed
  To investigate the strengths of this novel metric and interactive evaluation in comparison to state-of-the-art metrics and human evaluation of static conversations, we perform extended experiments with a set of models,…
• An Empirical Study on Context Length for Open-Domain Dialog Generation
  31 Aug 2024 1 repository listed
  We try to figure out how the choice of context length affects the model.
• Dior-CVAE: Pre-trained Language Models and Diffusion Priors for Variational Dialog Generation
  24 May 2023 1 repository listed
  These models also suffer from posterior collapse, i.
• Open-Domain Dialog Evaluation using Follow-Ups Likelihood
  12 Sep 2022 1 repository listed
  Automatic evaluation of open-domain dialogs remains an unsolved problem.
• Re2G: Retrieve, Rerank, Generate
  13 Jul 2022 1 repository listed Syntology ran 1 of 8 samples · 7 unverified
  As demonstrated by GPT-3 and T5, transformers grow in capability as parameter spaces become larger and larger.
• GODEL: Large-Scale Pre-Training for Goal-Directed Dialog
  22 Jun 2022 1 repository listed
  We introduce GODEL (Grounded Open Dialogue Language Model), a large pre-trained language model for dialog.
• CPED: A Large-Scale Chinese Personalized and Emotional Dialogue Dataset for Conversational AI
  29 May 2022 1 repository listed
  Finally, we provide baseline systems for these tasks and consider the function of speakers' personalities and emotions on conversation.
• InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning
  25 May 2022 1 repository listed Syntology ran 2 of 2 samples · 0 unverified
  We introduce InstructDial, an instruction tuning framework for dialogue, which consists of a repository of 48 diverse dialogue tasks in a unified text-to-text format created from 59 openly available dialogue datasets.
• What is wrong with you?: Leveraging User Sentiment for Automatic Dialog Evaluation
  25 Mar 2022 1 repository listed
  Existing model-based metrics for system response evaluation are trained on human annotated data, which is cumbersome to collect.
• Investigating Robustness of Dialog Models to Popular Figurative Language Constructs
  1 Oct 2021 1 repository listed
  Humans often employ figurative language use in communication, including during interactions with dialog systems.
• GenSF: Simultaneous Adaptation of Generative Pre-trained Models and Slot Filling
  13 Jun 2021 1 repository listed
  We instead achieve strong alignment by simultaneously modifying both the pre-trained model and the formulation of the downstream task, which is more efficient and preserves the scalability of transfer learning.
• Improving Automated Evaluation of Open Domain Dialog via Diverse Reference Augmentation
  5 Jun 2021 1 repository listed
  Multiple different responses are often plausible for a given open domain dialog context.
• HERALD: An Annotation Efficient Method to Detect User Disengagement in Social Conversations
  1 Jun 2021 1 repository listed Syntology ran 1 of 1 samples · 0 unverified
  Open-domain dialog systems have a user-centric goal: to provide humans with an engaging conversation experience.
• CoMAE: A Multi-factor Hierarchical Framework for Empathetic Response Generation
  18 May 2021 1 repository listed
  However, existing methods for empathetic response generation usually either consider only one empathy factor or ignore the hierarchical relationships between different factors, leading to a weak ability of empathy…
• Are Neural Open-Domain Dialog Systems Robust to Speech Recognition Errors in the Dialog History? An Empirical Study
  18 Aug 2020 1 repository listed
  Large end-to-end neural open-domain chatbots are becoming increasingly popular.
• Probing Neural Dialog Models for Conversational Understanding
  7 Jun 2020 1 repository listed
  The predominant approach to open-domain dialog generation relies on end-to-end training of neural models on chat datasets.
• Beyond User Self-Reported Likert Scale Ratings: A Comparison Model for Automatic Dialog Evaluation
  21 May 2020 1 repository listed Syntology ran 1 of 1 samples · 0 unverified
  Our experiments show that CMADE achieves 89.
• USR: An Unsupervised and Reference Free Evaluation Metric for Dialog Generation
  1 May 2020 1 repository listed
  The lack of meaningful automatic evaluation metrics for dialog has impeded open-domain dialog research.
• ClovaCall: Korean Goal-Oriented Dialog Speech Corpus for Automatic Speech Recognition of Contact Centers
  20 Apr 2020 1 repository listed
  Automatic speech recognition (ASR) via call is essential for various applications, including AI for contact center (AICC) services.
• Hierarchical Reinforcement Learning for Open-Domain Dialog
  17 Sep 2019 1 repository listed
  Open-domain dialog generation is a challenging problem; maximum likelihood training can lead to repetitive outputs, models have difficulty tracking long-term conversational goals, and training on standard movie or…
• A Multi-Turn Emotionally Engaging Dialog Model
  15 Aug 2019 1 repository listed
  Open-domain dialog systems (also known as chatbots) have increasingly drawn attention in natural language processing.
• Large-Scale Transfer Learning for Natural Language Generation
  1 Jul 2019 1 repository listed
  Large-scale pretrained language models define state of the art in natural language processing, achieving outstanding performance on a variety of tasks.
• Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog
  30 Jun 2019 1 repository listed Syntology ran 1 of 1 samples · 0 unverified
  Most deep reinforcement learning (RL) systems are not able to learn effectively from off-policy data, especially if they cannot explore online in the environment.
• Evaluating Coherence in Dialogue Systems using Entailment
  6 Apr 2019 1 repository listed Syntology ran 2 of 2 samples · 0 unverified · 1 pointer-only (licence)
  Evaluating open-domain dialogue systems is difficult due to the diversity of possible correct answers.

Syntology lines on 10 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.