{"url":"/dataset/comparative-question-completion","name":"Comparative Question Completion","full_name":null,"description_markdown":"**Comparative Question Completion** is a dataset to evaluate what do large Language Models learn.\r\n\r\nThe dataset includes short questions in natural language that make comparisons between entity pairs, for example, “is a cockroach or beetle more dangerous?”\r\n\r\nThe questions are in three subject domains: animals, cities and NBA players.\r\n\r\nIn each sentence, one of the compared entities in the sentence has been 'masked' (replaced with a [MASK] symbol). For example, for the question above the masked sentence is: “is a [MASK] or beetle more dangerous?” The dataset presents the task of automatically recovering the masked entity name, and provides the original entity for evaluation purposes. In addition to the original masked entity text (e.g., 'cockroach'), it details the respective Wikidata entity ID, (e.g., 'Q18123008').","description_withheld":null,"homepage":"https://github.com/google-research-datasets/comparative-question-completion","introduced_date":"2021-04-05","introduced_date_note":null,"introduced_by":{"paper":"/paper/what-s-the-best-place-for-an-ai-conference","title":"What's the best place for an AI conference, Vancouver or ______: Why completing comparative questions is difficult","first_author":"Avishai Zagoury","url":null},"license":{"name":"CC-BY 4.0","url":"https://github.com/google-research-datasets/comparative-question-completion/blob/main/LICENSE"},"modalities":[{"name":"Texts","url":"/datasets/modality/texts"}],"tasks":[{"name":"Language Modelling","url":"/task/language-modelling","datasets_with_task":"/datasets/task/language-modelling"}],"languages":[{"name":"English","url":"/datasets/language/english"}],"variants":["Comparative Question Completion"],"data_loaders":[{"repo":"https://github.com/google-research-datasets/comparative-question-completion","url":"https://github.com/google-research-datasets/comparative-question-completion","frameworks":[]}],"num_papers_in_archive":2,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[],"papers_with_a_benchmark_row":[],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}