{"url":"/dataset/convai2","name":"ConvAI2","full_name":"Conversational Intelligence Challenge 2","description_markdown":"The **ConvAI2** NeurIPS competition aimed at finding approaches to creating high-quality dialogue agents capable of meaningful open domain conversation. The ConvAI2 dataset for training models is based on the PERSONA-CHAT dataset. The speaker pairs each have assigned profiles coming from a set of 1155 possible personas (at training time), each consisting of at least 5 profile sentences, setting aside 100 never seen before personas for validation. As the original PERSONA-CHAT test set was released, a new hidden test set consisted of 100 new personas and over 1,015 dialogs was created by crowdsourced workers.\r\n\r\nTo avoid modeling that takes advantage of trivial word overlap, additional rewritten sets of the same train and test personas were crowdsourced, with related sentences that are rephrases, generalizations or specializations, rendering the task much more challenging. For example “I just got my nails done” is revised as “I love to pamper myself on a regular basis” and “I am on a diet now” is revised as “I need to lose weight.”\r\n\r\nThe training, validation and hidden test sets consists of 17,878, 1,000 and 1,015 dialogues, respectively.\r\n\r\nSource: [The Second Conversational Intelligence Challenge (ConvAI2)](https://paperswithcode.com/paper/the-second-conversational-intelligence/)\r\nImage Source: [The Second Conversational Intelligence Challenge (ConvAI2)](https://paperswithcode.com/paper/the-second-conversational-intelligence/)","description_withheld":null,"homepage":"https://parl.ai/projects/convai2/","introduced_date":"2019-01-01","introduced_date_note":null,"introduced_by":{"paper":"/paper/the-second-conversational-intelligence","title":"The Second Conversational Intelligence Challenge (ConvAI2)","first_author":"Emily Dinan","url":null},"license":{"name":"Custom","url":"http://convai.io/"},"modalities":[{"name":"Texts","url":"/datasets/modality/texts"},{"name":"Dialog","url":"/datasets/modality/dialog"}],"tasks":[{"name":"Visual Dialog","url":"/task/visual-dialogue","datasets_with_task":"/datasets/task/visual-dialogue"}],"languages":[{"name":"English","url":"/datasets/language/english"}],"variants":["ConvAI2"],"data_loaders":[{"repo":"https://github.com/huggingface/datasets","url":"https://huggingface.co/datasets/convai-challenge/conv_ai_2","frameworks":["tf","pytorch","jax"]},{"repo":"https://github.com/huggingface/datasets","url":"https://huggingface.co/datasets/conv_ai_2","frameworks":["tf","pytorch","jax"]}],"num_papers_in_archive":100,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[{"leaderboard":"/sota/visual-dialog-on-convai2","task":"Visual Dialog","dataset_variant":"ConvAI2","rows":1,"metrics":["BLEU-4","F1","ROUGE-L"],"first_row_in_archive_order":{"model":"Multi-Modal BlenderBot","paper":"/paper/multi-modal-open-domain-dialogue","metrics":{"BLEU-4":"1.1","F1":"18.4","ROUGE-L":"22.6"},"code_links":[]},"note":"rows are the archive's own order at snapshot; nothing here re-ranks them"}],"papers_with_a_benchmark_row":[{"paper":"/paper/multi-modal-open-domain-dialogue","title":"Multi-Modal Open-Domain Dialogue","date":"2020-10-02","rows_on_this_dataset":1,"code_links":0,"syntology":null}],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}