{"url":"/dataset/rwsd","name":"RWSD","full_name":"The Winograd Schema Challenge (Russian)","description_markdown":"A Winograd schema is a pair of sentences that differ in only one or two words and that contain an ambiguity that is resolved in opposite ways in the two sentences and requires the use of world knowledge and reasoning for its resolution. The schema takes its name from a well-known example by Terry Winograd.\r\n\r\nThe set would then be presented as a challenge for AI programs, along the lines of the Turing test. The strengths of the challenge are that it is clear-cut, in that the answer to each schema is a binary choice; vivid, in that it is obvious to non-experts that a program that fails to get the right answers clearly has serious gaps in its understanding; and difficult, in that it is far beyond the current state of the art.\r\n\r\n### Task Type\r\nLogic and Reasoning, World knowledge. Binary Classification: true/false\r\n\r\n### Example\r\n```\r\n{\r\n  \"text\": \"Кубок не помещается в коричневый чемодан, потому что он слишком большой.\"\r\n  \"label\": false,\r\n  \"idx\": 5,\r\n  \"target\": {\r\n    \"span1_text\": \"чемодан\",\r\n    \"span2_text\": \"он слишком большой\",\r\n    \"span1_index\": 5,\r\n    \"span2_index\": 8\r\n  },\r\n}\r\n```\r\n### How did we collect data? \r\nAll text examples were collected manually translating and adapting original Winograd dataset for Russian. Human assessment was carried out on Yandex.Toloka.","description_withheld":null,"homepage":"https://github.com/RussianNLP/RussianSuperGLUE","introduced_date":"2020-10-29","introduced_date_note":null,"introduced_by":{"paper":"/paper/russiansuperglue-a-russian-language","title":"RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark","first_author":"Tatiana Shavrina","url":null},"license":{"name":"MIT License","url":"https://github.com/RussianNLP/RussianSuperGLUE/blob/master/LICENSE"},"modalities":[{"name":"Texts","url":"/datasets/modality/texts"}],"tasks":[{"name":"Common Sense Reasoning","url":"/task/common-sense-reasoning","datasets_with_task":"/datasets/task/common-sense-reasoning"},{"name":"Logical Reasoning Reading Comprehension","url":"/task/logical-reasoning-reading-comprehension","datasets_with_task":"/datasets/task/logical-reasoning-reading-comprehension"}],"languages":[{"name":"Russian","url":"/datasets/language/russian"}],"variants":["RWSD"],"data_loaders":[],"num_papers_in_archive":7,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[{"leaderboard":"/sota/common-sense-reasoning-on-rwsd","task":"Common Sense Reasoning","dataset_variant":"RWSD","rows":22,"metrics":["Accuracy"],"first_row_in_archive_order":{"model":"Golden Transformer","paper":null,"metrics":{"Accuracy":"0.545"},"code_links":[]},"note":"rows are the archive's own order at snapshot; nothing here re-ranks them"}],"papers_with_a_benchmark_row":[{"paper":"/paper/unreasonable-effectiveness-of-rule-based","title":"Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks","date":"2021-05-03","rows_on_this_dataset":3,"code_links":0,"syntology":null},{"paper":"/paper/russiansuperglue-a-russian-language","title":"RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark","date":"2020-10-29","rows_on_this_dataset":2,"code_links":2,"syntology":{"read_at":"2026-09-24T18:15:14+00:00","samples_harvested":1,"samples_ran":1,"samples_unverified":0,"pointer_only_for_licence":0,"claim":"Per-sample execution on synthesized fixtures; not a correctness claim."}},{"paper":"/paper/mt5-a-massively-multilingual-pre-trained-text","title":"mT5: A massively multilingual pre-trained text-to-text transformer","date":"2020-10-22","rows_on_this_dataset":1,"code_links":8,"syntology":{"read_at":"2026-09-24T18:15:14+00:00","samples_harvested":13,"samples_ran":0,"samples_unverified":13,"pointer_only_for_licence":0,"claim":"Per-sample execution on synthesized fixtures; not a correctness claim."}}],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":2,"samples_harvested":14,"samples_ran":1,"samples_unverified":13,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":1,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}