{"url":"/dataset/rucos","name":"RuCoS","full_name":"Russian Reading Comprehension with Commonsense Reasoning","description_markdown":"Russian reading comprehension with Commonsense reasoning (RuCoS) is a large-scale reading comprehension dataset that requires commonsense reasoning. RuCoS consists of queries automatically generated from CNN/Daily Mail news articles; the answer to each query is a text span from a summarizing passage of the corresponding news. The goal of RuCoS is to evaluate a machine`s ability of commonsense reasoning in reading comprehension.\r\n\r\n\r\nExample\r\n```\r\n  {'source': 'Lenta',\r\n   'passage': {\r\n          'text':\r\n          'Мать двух мальчиков, брошенных отцом в московском аэропорту Шереметьево, забрала их. Об этом сообщили ТАСС в пресс-службе министерства образования и науки Хабаровского края. Сейчас младший ребенок посещает детский сад, а старший ходит в школу. В учебных заведениях с ними по необходимости работают штатные психологи. Также министерство социальной защиты населения рассматривает вопрос о бесплатном оздоровлении детей в летнее время. Через несколько дней после того, как Виктор Гаврилов бросил своих детей в аэропорту, он явился с повинной к следователям в городе Батайске Ростовской области.\\n@context\\nБросившего детей в Шереметьево отца задержали за насилие над женой\\n@context\\nРоссиянина заподозрили в истязании брошенных в Шереметьево детей\\n@context\\nОставивший двоих детей в Шереметьево россиянин сам пришел к следователям',\r\n          'entities': [\r\n              {'start': 60, 'end': 71, 'text': 'Шереметьево'},\r\n              {'start': 102, 'end': 106, 'text': 'ТАСС'},\r\n              {'start': 155, 'end': 172, 'text': 'Хабаровского края'},\r\n              {'start': 470, 'end': 485, 'text': 'Виктор Гаврилов'},\r\n              {'start': 563, 'end': 571, 'text': 'Батайске'},\r\n              {'start': 572, 'end': 590, 'text': 'Ростовской области'},\r\n              {'start': 620, 'end': 631, 'text': 'Шереметьево'},\r\n              {'start': 725, 'end': 736, 'text': 'Шереметьево'},\r\n              {'start': 778, 'end': 789, 'text': 'Шереметьево'}\r\n          ]\r\n      },\r\n      'qas': [\r\n          {\r\n              'query': '26 января @placeholder бросил сыновей в возрасте пяти и семи лет в Шереметьево.',\r\n              'answers': [\r\n                  {'start': 470, 'end': 485, 'text': 'Виктор Гаврилов'}\r\n              ],\r\n              'idx': 0\r\n          }\r\n      ],\r\n      'idx': 0\r\n  }\r\n```\r\n\r\n### How did we collect data? \r\nAll text examples were collected from open news sources, then automatically filtered with QA systems to prevent obvious questions to infiltrate the dataset. The texts were then filtered by IPM frequency of the contained words and, finally, manually reviewed.","description_withheld":null,"homepage":"https://github.com/RussianNLP/RussianSuperGLUE","introduced_date":"2020-06-11","introduced_date_note":null,"introduced_by":{"paper":"/paper/russiansuperglue-a-russian-language","title":"RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark","first_author":"Tatiana Shavrina","url":null},"license":{"name":"MIT License","url":"https://github.com/RussianNLP/RussianSuperGLUE/blob/master/LICENSE"},"modalities":[{"name":"Texts","url":"/datasets/modality/texts"}],"tasks":[{"name":"Question Answering","url":"/task/question-answering","datasets_with_task":"/datasets/task/question-answering"},{"name":"Common Sense Reasoning","url":"/task/common-sense-reasoning","datasets_with_task":"/datasets/task/common-sense-reasoning"},{"name":"Machine Reading Comprehension","url":"/task/machine-reading-comprehension","datasets_with_task":"/datasets/task/machine-reading-comprehension"}],"languages":[{"name":"Russian","url":"/datasets/language/russian"}],"variants":["RuCoS"],"data_loaders":[],"num_papers_in_archive":5,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[{"leaderboard":"/sota/common-sense-reasoning-on-rucos","task":"Common Sense Reasoning","dataset_variant":"RuCoS","rows":22,"metrics":["Average F1","EM "],"first_row_in_archive_order":{"model":"Human Benchmark","paper":"/paper/russiansuperglue-a-russian-language","metrics":{"Average F1":"0.93","EM ":"0.89"},"code_links":[{"title":"RussianNLP/RussianSuperGLUE","url":"https://github.com/RussianNLP/RussianSuperGLUE"},{"title":"RussianNLP/MOROCCO","url":"https://github.com/RussianNLP/MOROCCO"}]},"note":"rows are the archive's own order at snapshot; nothing here re-ranks them"}],"papers_with_a_benchmark_row":[{"paper":"/paper/unreasonable-effectiveness-of-rule-based","title":"Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks","date":"2021-05-03","rows_on_this_dataset":3,"code_links":0,"syntology":null},{"paper":"/paper/russiansuperglue-a-russian-language","title":"RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark","date":"2020-10-29","rows_on_this_dataset":2,"code_links":2,"syntology":{"read_at":"2026-09-24T18:15:14+00:00","samples_harvested":1,"samples_ran":1,"samples_unverified":0,"pointer_only_for_licence":0,"claim":"Per-sample execution on synthesized fixtures; not a correctness claim."}},{"paper":"/paper/mt5-a-massively-multilingual-pre-trained-text","title":"mT5: A massively multilingual pre-trained text-to-text transformer","date":"2020-10-22","rows_on_this_dataset":1,"code_links":8,"syntology":{"read_at":"2026-09-24T18:15:14+00:00","samples_harvested":13,"samples_ran":0,"samples_unverified":13,"pointer_only_for_licence":0,"claim":"Per-sample execution on synthesized fixtures; not a correctness claim."}}],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":2,"samples_harvested":14,"samples_ran":1,"samples_unverified":13,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":1,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}