Datasets › WSC
WSC (Winograd Schema Challenge)
The Winograd Schema Challenge was introduced both as an alternative to the Turing Test and as a test of a system’s ability to do commonsense reasoning. A Winograd schema is a pair of sentences differing in one or two words with a highly ambiguous pronoun, resolved differently in the two sentences, that appears to require commonsense knowledge to be resolved correctly. The examples were designed to be easily solvable by humans but difficult for machines, in principle requiring a deep understanding of the content of the text and the situation it describes.
The original Winograd Schema Challenge dataset consisted of 100 Winograd schemas constructed manually by AI experts. As of 2020 there are 285 examples available; however, the last 12 examples were only added recently. To ensure consistency with earlier models, several authors often prefer to report the performance on the first 273 examples only. These datasets are usually referred to as WSC285 and WSC273, respectively.
Source: https://arxiv.org/pdf/2004.13831.pdf Image Source: https://arxiv.org/pdf/1907.11983.pdf
Benchmarks archive 2025-07-28
All 2 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.
| First row (archive order) | Paper | Code | ||||
|---|---|---|---|---|---|---|
| Coreference Resolution | Winograd Schema Challenge | PaLM 540B (fine-tuned) Accuracy 100 | PaLM: Scaling Language Modeling with Pathways | lucidrains/CoCa-pytorch +6 | 82 | Compare |
| Classification | WSC | OPT-1.3B Test Accuracy 64.16% | Achieving Dimension-Free Communication in Federated... | ZidongLiu/DeComFL | 2 | Compare |
Papers archive 2025-07-28
30 shown of 38 papers with a leaderboard row on this dataset's benchmarks, newest first. The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset; the archive's count for this dataset is 361. The Syntology column is from Syntology's graph (read 2026-09-24), stated per sample; it is not part of any archive number.
The full list of 38 is in the JSON twin.
Dataset loaders archive 2025-07-28
6 loaders as listed in the archive; links are outbound and not re-checked here.
Tasks archive 2025-07-28
License archive 2025-07-28
Modalities archive 2025-07-28
Languages archive 2025-07-28
Variants archive 2025-07-28
- Winograd Schema Challenge
- WSC
- winograd_wsc
3 variant names, as the archive lists them.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections