PIQA (Physical Interaction: Question Answering)
Introduced by Yonatan Bisk et al. in
PIQA: Reasoning about Physical Commonsense in Natural Language 26 Nov 2019 archive 2025-07-28
PIQA is a dataset for commonsense reasoning, and was created to investigate the physical knowledge of existing models in NLP.
Source: PIQA
Benchmarks archive 2025-07-28
All 2 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.
Syntology 59 newer papers were judged to report on this dataset's benchmarks so far (Claude Sonnet 4.5, reading their tables: a model's judgement, not a result; 9 placed): Question Answering · PIQA (59 judged, 9 placed). Drawn from 8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read).
Papers archive 2025-07-28
papers with a benchmark row: 28 · with a code link: 26 · where Syntology ran a sample: 16 (16 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology
Show:
all papers with a benchmark row only where code ran (16 of 28 with a benchmark row: 16 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 28 of 28 with a leaderboard row on this dataset's benchmarks, newest first by the archive's date (ties by slug; undated papers last).
The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset, not that list; the archive's count for this dataset is 772. The Syntology column is from Syntology's graph, stated per sample; it is not part of any archive number. A line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified” (C is a part of N, never taken away from it); the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. When the archive marks a repository official for the paper, the cell starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. Syntology's record for this page last changed 2026-10-05 ; when this build read Syntology's graph is in the build record .
Paper Code Results Date Samples run Syntology
SHAKTI: A 2.5 Billion Parameter Small Language Model Optimized for Edge AI and Low-Resource Environments
0
1
15 Oct 2024
not harvested
Mixture-of-Subspaces in Low-Rank Adaptation
1
1
16 Jun 2024
official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (6 pointer-only for licence )
MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts
2
3
22 Apr 2024
official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 1 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 5 unverified
Mixtral of Experts
6
2
8 Jan 2024
5 ran (of which 5 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 5 samples that ran constructed an object rather than computing a result
Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks
2
1
5 Jan 2024
not harvested
Mistral 7B
6
1
10 Oct 2023
official (archive's flag): 9 ran · 10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (2 pointer-only for licence )
Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
2
3
10 Oct 2023
official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified
Textbooks Are All You Need II: phi-1.5 technical report
1
1
11 Sep 2023
not harvested
Llama 2: Open Foundation and Fine-Tuned Chat Models
19
4
18 Jul 2023
community repositories only · 33 ran (of which 7 constructed an object rather than computing a result; 20 with no instrument failure: 1 honoured, 1 violated, 18 with no contract checked; 13 where Syntology's instrument failed) · 19 unverified (20 pointer-only for licence )
PaLM 2 Technical Report
1
3
17 May 2023
not harvested
LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions
1
6
27 Apr 2023
not harvested
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
4
4
3 Apr 2023
not harvested
BloombergGPT: A Large Language Model for Finance
2
4
30 Mar 2023
not harvested
LLaMA: Open and Efficient Foundation Language Models
57
4
27 Feb 2023
official: no sample here; runs from other or unrecorded repositories · 37 ran (of which 9 constructed an object rather than computing a result; 25 with no instrument failure: 3 honoured, 0 violated, 22 with no contract checked; 12 where Syntology's instrument failed) · 21 unverified (4 pointer-only for licence )
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot
6
5
2 Jan 2023
official (archive's flag): 2 ran · 9 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified (9 pointer-only for licence )
Two is Better than Many? Binary Classification as an Effective Approach to Multi-Choice Question Answering
1
2
29 Oct 2022
official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 5 unverified
Task Compass: Scaling Multi-task Pre-training with Task Prefix
1
3
12 Oct 2022
not harvested
Training Compute-Optimal Large Language Models
2
1
29 Mar 2022
8 ran (of which 3 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified (4 pointer-only for licence )
Efficient Language Modeling with Sparse all-MLP
0
4
14 Mar 2022
not harvested
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
3
1
8 Dec 2021
not harvested
Finetuned Language Models Are Zero-Shot Learners
8
2
3 Sep 2021
official: harvested for another paper · 0 ran · 1 unverified
UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark
1
1
24 Mar 2021
official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified
Language Models are Few-Shot Learners
67
2
28 May 2020
community repositories only · 45 ran (of which 0 constructed an object rather than computing a result; 37 with no instrument failure: 3 honoured, 2 violated, 32 with no contract checked; 8 where Syntology's instrument failed) · 20 unverified (7 pointer-only for licence )
UnifiedQA: Crossing Format Boundaries With a Single QA System
2
1
2 May 2020
official (archive's flag): 4 ran · 4 ran (of which 3 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 1 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (1 pointer-only for licence )
PIQA: Reasoning about Physical Commonsense in Natural Language
2
4
26 Nov 2019
not harvested
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
10
1
17 Sep 2019
community repositories only · 12 ran (of which 2 constructed an object rather than computing a result; 7 with no instrument failure: 4 honoured, 0 violated, 3 with no contract checked; 5 where Syntology's instrument failed) · 35 unverified (15 pointer-only for licence )
RoBERTa: A Robustly Optimized BERT Pretraining Approach
67
1
26 Jul 2019
community repositories only · 37 ran (of which 11 constructed an object rather than computing a result; 36 with no instrument failure: 2 honoured, 0 violated, 34 with no contract checked; 1 where Syntology's instrument failed) · 11 unverified (24 pointer-only for licence )
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
534
1
11 Oct 2018
official: no sample here; runs from other or unrecorded repositories · 300 ran (of which 75 constructed an object rather than computing a result; 234 with no instrument failure: 18 honoured, 5 violated, 211 with no contract checked; 66 where Syntology's instrument failed) · 359 unverified (164 pointer-only for licence )
Dataset loaders archive 2025-07-28
7 loaders as listed in the archive; links are outbound and not re-checked here.
Tasks archive 2025-07-28
License archive 2025-07-28
Unknown
Modalities archive 2025-07-28
Languages archive 2025-07-28
No language tagged.
Variants archive 2025-07-28
1 variant name, as the archive lists them.