Datasets › WinoGrande

WinoGrande

Introduced by Keisuke Sakaguchi et al. in WinoGrande: An Adversarial Winograd Schema Challenge at Scale archive 2025-07-28

WinoGrande is a large-scale dataset of 44k problems, inspired by the original WSC design, but adjusted to improve both the scale and the hardness of the dataset. The key steps of the dataset construction consist of (1) a carefully designed crowdsourcing procedure, followed by (2) systematic bias reduction using a novel AfLite algorithm that generalizes human-detectable word associations to machine-detectable embedding associations.

Source: WinoGrande: An Adversarial Winograd Schema Challenge at Scale Image Source: https://winogrande.allenai.org/

Benchmarks archive 2025-07-28

All 7 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.

Syntology 33 newer papers were judged to report on this dataset's benchmarks so far (Claude Sonnet 4.5, reading their tables: a model's judgement, not a result; 1 placed): Common Sense Reasoning · WinoGrande (31 judged, 1 placed), parameter-efficient fine-tuning · WinoGrande (8 judged, 0 placed). Drawn from 8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read).

First row (archive order)PaperCode
Common Sense Reasoning WinoGrande ST-MoE-32B 269B (fine-tuned) Accuracy 96.1 ST-MoE: Designing Stable and Transferable Sparse Expert Models tensorflow/mesh +2 77 Compare
parameter-efficient fine-tuning WinoGrande LLaMA2-7b Accuracy (% ) 70.80 GIFT-SW: Gaussian noise Injected Fine-Tuning of Salient... On-Point-RND/GIFT_SW 3 Compare
Text Generation WinoGrande no rows — — 0 Compare
Text Generation Winogrande (5-shot) no rows — — 0 Compare
Text Generation Winogrande TR no rows — — 0 Compare
Text Generation Winogrande TR v0.2 no rows — — 0 Compare
Winogrande WinoGrande no rows — — 0 Compare

Papers archive 2025-07-28

papers with a benchmark row: 34 · with a code link: 30 · where Syntology ran a sample: 17 (14 with a run with no instrument failure, 3 where every run was a failure of Syntology's instrument) Syntology

Show: all papers with a benchmark rowonly where code ran (17 of 34 with a benchmark row: 14 with a run with no instrument failure, 3 where every run was a failure of Syntology's instrument)

Page 1 of 1: papers 1 to 34 of 34 with a leaderboard row on this dataset's benchmarks, newest first by the archive's date (ties by slug; undated papers last).

The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset, not that list; the archive's count for this dataset is 703. The Syntology column is from Syntology's graph, stated per sample; it is not part of any archive number. A line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified” (C is a part of N, never taken away from it); the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. When the archive marks a repository official for the paper, the cell starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record.

PaperCodeResultsDateSamples run Syntology
GIFT-SW: Gaussian noise Injected Fine-Tuning of Salient Weights for LLMs 1 1 27 Aug 2024 official (archive's flag): 2 ran · 5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (5 pointer-only for licence)
Mixture-of-Subspaces in Low-Rank Adaptation 1 1 16 Jun 2024 official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (6 pointer-only for licence)
MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts 2 3 22 Apr 2024 official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 1 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 5 unverified
Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM 1 1 12 Mar 2024 not harvested
The Claude 3 Model Family: Opus, Sonnet, Haiku 0 3 4 Mar 2024 not harvested
DoRA: Weight-Decomposed Low-Rank Adaptation 5 1 14 Feb 2024 official (archive's flag): 7 ran · 8 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 8 where Syntology's instrument failed) · 7 unverified (14 pointer-only for licence)
Mixtral of Experts 6 2 8 Jan 2024 5 ran (of which 5 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 5 samples that ran constructed an object rather than computing a result
Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks 2 1 5 Jan 2024 not harvested
Mistral 7B 6 1 10 Oct 2023 official (archive's flag): 9 ran · 10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (2 pointer-only for licence)
Textbooks Are All You Need II: phi-1.5 technical report 1 1 11 Sep 2023 not harvested
The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning 2 1 23 May 2023 not harvested
PaLM 2 Technical Report 1 3 17 May 2023 not harvested
LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions 1 6 27 Apr 2023 not harvested
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling 4 4 3 Apr 2023 not harvested
BloombergGPT: A Large Language Model for Finance 2 4 30 Mar 2023 not harvested
GPT-4 Technical Report 11 2 15 Mar 2023 community repositories only · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 2 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (1 pointer-only for licence)
LLaMA: Open and Efficient Foundation Language Models 57 4 27 Feb 2023 official: no sample here; runs from other or unrecorded repositories · 37 ran (of which 9 constructed an object rather than computing a result; 25 with no instrument failure: 3 honoured, 0 violated, 22 with no contract checked; 12 where Syntology's instrument failed) · 21 unverified (4 pointer-only for licence)
Exploring the Benefits of Training Expert Language Models over Instruction Tuning 2 1 7 Feb 2023 official (archive's flag): 2 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (3 pointer-only for licence)
Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models 0 1 28 Oct 2022 not harvested
Task Compass: Scaling Multi-task Pre-training with Task Prefix 1 3 12 Oct 2022 not harvested
Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learners 1 1 6 Oct 2022 not harvested
PaLM: Scaling Language Modeling with Pathways 7 3 5 Apr 2022 32 ran (of which 16 constructed an object rather than computing a result; 24 with no instrument failure: 2 honoured, 1 violated, 21 with no contract checked; 8 where Syntology's instrument failed) · 5 unverified
Training Compute-Optimal Large Language Models 2 1 29 Mar 2022 8 ran (of which 3 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified (4 pointer-only for licence)
Efficient Language Modeling with Sparse all-MLP 0 5 14 Mar 2022 not harvested
ST-MoE: Designing Stable and Transferable Sparse Expert Models 3 2 17 Feb 2022 community repositories only · 5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 3 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (5 pointer-only for licence)
Scaling Language Models: Methods, Analysis & Insights from Training Gopher 3 1 8 Dec 2021 not harvested
Finetuned Language Models Are Zero-Shot Learners 8 2 3 Sep 2021 official: harvested for another paper · 0 ran · 1 unverified
LoRA: Low-Rank Adaptation of Large Language Models 74 1 17 Jun 2021 community repositories only · 51 ran (of which 19 constructed an object rather than computing a result; 43 with no instrument failure: 1 honoured, 0 violated, 42 with no contract checked; 8 where Syntology's instrument failed) · 33 unverified (30 pointer-only for licence)
Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schema 0 7 16 Apr 2021 not harvested
UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark 1 1 24 Mar 2021 official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified
Language Models are Few-Shot Learners 67 2 28 May 2020 community repositories only · 45 ran (of which 0 constructed an object rather than computing a result; 37 with no instrument failure: 3 honoured, 2 violated, 32 with no contract checked; 8 where Syntology's instrument failed) · 20 unverified (7 pointer-only for licence)
UnifiedQA: Crossing Format Boundaries With a Single QA System 2 2 2 May 2020 official (archive's flag): 4 ran · 4 ran (of which 3 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 1 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (1 pointer-only for licence)
Generative Data Augmentation for Commonsense Reasoning 1 1 24 Apr 2020 official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (4 pointer-only for licence)
WinoGrande: An Adversarial Winograd Schema Challenge at Scale 10 6 24 Jul 2019 not harvested

Dataset loaders archive 2025-07-28

6 loaders as listed in the archive; links are outbound and not re-checked here.

Tasks archive 2025-07-28

License archive 2025-07-28

CC-BY

Modalities archive 2025-07-28

Languages archive 2025-07-28

No language tagged.

Variants archive 2025-07-28

  • WinoGrande
  • Winogrande
  • Winogrande (5-shot)
  • Winogrande TR v0.2
  • Winogrande TR

5 variant names, as the archive lists them.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections