{"url":"/dataset/xwino","name":"XWINO","full_name":null,"description_markdown":"XWINO is a multilingual collection of Winograd Schemas in six languages that can be used for evaluation of cross-lingual\r\ncommonsense reasoning capabilities. \r\n\r\nThe datasets that comprise XWINO are:\r\n\r\n * Source: The original [Winograd Schema Challenge](https://cs.nyu.edu/~davise/papers/WinogradSchemas/WSCollection.xml) ([Levesque](http://www.cs.toronto.edu/~hector/Papers/winograd.pdf), 2012);\r\n * Source: Additional data from the [SuperGLUE](https://super.gluebenchmark.com/tasks/) WSC benchmark ([Wang et al.](https://papers.nips.cc/paper/2019/hash/4496bf24afe7fab6f046bf4923da8de6-Abstract.html), 2019);\r\n * Source: The [Definite Pronoun Resolution](http://www.hlt.utdallas.edu/~vince/data/emnlp12/) dataset ([Rahman and Ng](https://www.aclweb.org/anthology/D12-1071/), 2012) (accessed from https://github.com/Yre/wsc_naive);\r\n * Source: A collection of [French Winograd Schemas](http://www.llf.cnrs.fr/fr/winograd-fr) ([Amsili and Seminck](https://www.aclweb.org/anthology/W17-1504/), 2017);\r\n * Source: [Japanese translation](https://github.com/ku-nlp/Winograd-Schema-Challenge-Ja) of Winograd Schema Challenge ([柴田知秀 et al.](http://www.anlp.jp/proceedings/annual_meeting/2015/pdf_dir/E3-1.pdf), 2015);\r\n *  Source: [Russian Winograd Schema Challenge](https://russiansuperglue.com/tasks/task_info/RWSD) ([Shavrina et al.](https://www.aclweb.org/anthology/2020.emnlp-main.381/), 2020);\r\n * Source: A collection of [Winograd Schemas in Chinese](https://cs.nyu.edu/~davise/papers/WinogradSchemas/WSChinese.html);\r\n * Source: Winograd Schemas [in Portuguese](https://github.com/gabimelo/portuguese_wsc) ([Melo et al.](https://www.teses.usp.br/teses/disponiveis/3/3141/tde-14012021-124730/es.php), 2019).","description_withheld":null,"homepage":"https://github.com/yandex-research/crosslingual_winograd","introduced_date":"2021-06-22","introduced_date_note":null,"introduced_by":{"paper":"/paper/it-s-all-in-the-heads-using-attention-heads","title":"It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoning","first_author":"Alexey Tikhonov","url":null},"license":{"name":"Apache-2.0 License","url":"https://github.com/yandex-research/crosslingual_winograd/blob/main/LICENSE"},"modalities":[{"name":"Texts","url":"/datasets/modality/texts"}],"tasks":[{"name":"Natural Language Inference","url":"/task/natural-language-inference","datasets_with_task":"/datasets/task/natural-language-inference"},{"name":"Coreference Resolution","url":"/task/coreference-resolution","datasets_with_task":"/datasets/task/coreference-resolution"},{"name":"Natural Language Understanding","url":"/task/natural-language-understanding","datasets_with_task":"/datasets/task/natural-language-understanding"}],"languages":[{"name":"English","url":"/datasets/language/english"},{"name":"French","url":"/datasets/language/french"},{"name":"Chinese","url":"/datasets/language/chinese"},{"name":"Japanese","url":"/datasets/language/japanese"},{"name":"Russian","url":"/datasets/language/russian"},{"name":"Portuguese","url":"/datasets/language/portuguese"}],"variants":["XWINO"],"data_loaders":[],"num_papers_in_archive":4,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[{"leaderboard":"/sota/natural-language-inference-on-xwino","task":"Natural Language Inference","dataset_variant":"XWINO","rows":1,"metrics":["Accuracy"],"first_row_in_archive_order":{"model":"mGPT","paper":"/paper/mgpt-few-shot-learners-go-multilingual","metrics":{"Accuracy":"0.562"},"code_links":[{"title":"ai-forever/mgpt","url":"https://github.com/ai-forever/mgpt"}]},"note":"rows are the archive's own order at snapshot; nothing here re-ranks them"}],"papers_with_a_benchmark_row":[{"paper":"/paper/mgpt-few-shot-learners-go-multilingual","title":"mGPT: Few-Shot Learners Go Multilingual","date":"2022-04-15","rows_on_this_dataset":1,"code_links":1,"syntology":null}],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}