Papers › Training End-to-End Dialogue Systems with the Ubuntu Dialogue Corpus
Training End-to-End Dialogue Systems with the Ubuntu Dialogue Corpus
Ryan Lowe, Nissan Pow, Iulian Vlad Serban, Laurent Charlin, Chia-Wei Liu, Joelle Pineau
In this paper, we analyze neural network-based dialogue systems trained in an end-to-end manner using an updated version of the recent Ubuntu Dialogue Corpus, a dataset containing almost 1 million multi-turn dialogues, with a total of over 7 million utterances and 100 million words. This dataset is interesting because of its size, long context lengths, and technical nature; thus, it can be used to train large models directly from data with minimal feature engineering. We provide baselines in two different environments: one where models are trained to select the correct next response from a list of candidate responses, and one where models are trained to maximize the loglikelihood of a generated utterance conditioned on the context of the conversation. These are both evaluated on a recall task that we call next utterance classification (NUC), and using vector-based metrics that capture the topicality of the responses. We observe that current end-to-end models are unable to completely solve these tasks; thus, we provide a qualitative error analysis to determine the primary causes of error for end-to-end models evaluated on NUC, and examine sample utterances from the generative models. As a result of this analysis, we suggest some promising directions for future research on the Ubuntu Dialogue Corpus, which can also be applied to end-to-end dialogue systems in general.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Conversation Disentanglement | Linux IRC (Ch2 Elsner) | Heuristic | 1-1 | 45.1 | #3 of 3 | Archive leaderboard | report |
| Conversation Disentanglement | Linux IRC (Ch2 Elsner) | Heuristic | Local | 73.8 | #3 of 3 | Archive leaderboard | report |
| Conversation Disentanglement | Linux IRC (Ch2 Elsner) | Heuristic | Shen F-1 | 51.8 | #3 of 3 | Archive leaderboard | report |
| Conversation Disentanglement | Linux IRC (Ch2 Kummerfeld) | Heuristic | 1-1 | 43.4 | #3 of 3 | Archive leaderboard | report |
| Conversation Disentanglement | Linux IRC (Ch2 Kummerfeld) | Heuristic | Local | 67.9 | #3 of 3 | Archive leaderboard | report |
| Conversation Disentanglement | Linux IRC (Ch2 Kummerfeld) | Heuristic | Shen F-1 | 50.7 | #3 of 3 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections