{"url":"/dataset/rrs-ranking-test","name":"RRS Ranking Test","full_name":"Restoration-200k for Response Selection with Ranking Test Set","description_markdown":"|           | Train | Validation | Test    | Ranking Test |\r\n| --------- | ----- | ---------- | ------- | ------------ |\r\n| size      | 0.4M  | 50K        | 5K      | 800          |\r\n| pos:neg   | 1:1   | 1:9        | 1.2:8.8 | -            |\r\n| avg turns | 5.0   | 5.0        | 5.0     | 5.0          |\r\n\r\nRanking test set contains the high-quality responses that selected by some baselines, and their correlation with the conversation context are carefully annotated by 8 professional annotators (the average annotation scores are saved for ranking). For ranking test set, the metrics should be NDCG@3 and NDCG@5, since the correlation scores are provided. More details are available in the Appendix of the paper.","description_withheld":null,"homepage":"https://github.com/gmftbyGMFTBY/SimpleReDial-v1","introduced_date":"2021-10-13","introduced_date_note":null,"introduced_by":{"paper":"/paper/exploring-dense-retrieval-for-dialogue","title":"Exploring Dense Retrieval for Dialogue Response Selection","first_author":"Tian Lan","url":null},"license":null,"modalities":[{"name":"Texts","url":"/datasets/modality/texts"}],"tasks":[{"name":"Conversational Response Selection","url":"/task/conversational-response-selection","datasets_with_task":"/datasets/task/conversational-response-selection"}],"languages":[{"name":"Chinese","url":"/datasets/language/chinese"}],"variants":["RRS Ranking Test"],"data_loaders":[],"num_papers_in_archive":5,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[{"leaderboard":"/sota/conversational-response-selection-on-rrs-1","task":"Conversational Response Selection","dataset_variant":"RRS Ranking Test","rows":4,"metrics":["NDCG@3","NDCG@5"],"first_row_in_archive_order":{"model":"Poly-encoder","paper":"/paper/190501969","metrics":{"NDCG@3":"0.679","NDCG@5":"0.765"},"code_links":[{"title":"sfzhou5678/PolyEncoder","url":"https://github.com/sfzhou5678/PolyEncoder"},{"title":"chijames/Poly-Encoder","url":"https://github.com/chijames/Poly-Encoder"},{"title":"csong27/collision-bert","url":"https://github.com/csong27/collision-bert"},{"title":"llStringll/Poly-encoders","url":"https://github.com/llStringll/Poly-encoders"},{"title":"fangrouli/Document-embedding-generation-models","url":"https://github.com/fangrouli/Document-embedding-generation-models"},{"title":"i2r-simmc/i2r-simmc-2020","url":"https://github.com/i2r-simmc/i2r-simmc-2020"},{"title":"Alexey-Borisov/3_course_diary","url":"https://github.com/Alexey-Borisov/3_course_diary"}]},"note":"rows are the archive's own order at snapshot; nothing here re-ranks them"}],"papers_with_a_benchmark_row":[{"paper":"/paper/fine-grained-post-training-for-improving","title":"Fine-grained Post-training for Improving Retrieval-based Dialogue Systems","date":"2021-05-24","rows_on_this_dataset":1,"code_links":1,"syntology":null},{"paper":"/paper/speaker-aware-bert-for-multi-turn-response","title":"Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots","date":"2020-04-07","rows_on_this_dataset":1,"code_links":2,"syntology":null},{"paper":"/paper/domain-adaptive-training-bert-for-response","title":"An Effective Domain Adaptive Post-Training Method for BERT in Response Selection","date":"2019-08-13","rows_on_this_dataset":1,"code_links":1,"syntology":null},{"paper":"/paper/190501969","title":"Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring","date":"2019-04-22","rows_on_this_dataset":1,"code_links":7,"syntology":null}],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}