Datasets › GrailQA
GrailQA (Strongly Generalizable Question Answering)
GrailQA is a new large-scale, high-quality dataset for question answering on knowledge bases (KBQA) on Freebase with 64,331 questions annotated with both answers and corresponding logical forms in different syntax (i.e., SPARQL, S-expression, etc.). It can be used to test three levels of generalization in KBQA: i.i.d., compositional, and zero-shot.
Benchmarks archive 2025-07-28
All 4 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.
| First row (archive order) | Paper | Code | ||||
|---|---|---|---|---|---|---|
| Question Generation | GrailQA-Compositional | FactJointGT METEOR 36.21 | FactSpotter: Evaluating the Factual Faithfulness of... | guihuzhang/FactSpotter | 4 | Compare |
| Question Generation | GrailQA-IID | FactT5B BLEU 46.10 | FactSpotter: Evaluating the Factual Faithfulness of... | guihuzhang/FactSpotter | 4 | Compare |
| Question Generation | GrailQA-Zero-Shot | JointGT METEOR 37.69 | JointGT: Graph-Text Joint Representation Learning for... | thu-coai/JointGT | 4 | Compare |
| Knowledge Base Question Answering | GrailQA | ReTraCk Compositional EM 61.5 | ReTraCk: A Flexible and Efficient Framework for... | microsoft/KC | 2 | Compare |
Papers archive 2025-07-28
5 shown of 5 papers with a leaderboard row on this dataset's benchmarks, newest first. The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset; the archive's count for this dataset is 34. The Syntology column is from Syntology's graph (read 2026-09-24), stated per sample; it is not part of any archive number.
| Date | Samples run Syntology | |||
|---|---|---|---|---|
| Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoning | 1 | 1 | 18 Oct 2024 | ran 1 of 12 samples (11 unverified) |
| FactSpotter: Evaluating the Factual Faithfulness of Graph-to-Text Generation | 1 | 6 | 25 Oct 2023 | not harvested |
| ReTraCk: A Flexible and Efficient Framework for Knowledge Base Question Answering | 1 | 1 | 1 Aug 2021 | not harvested |
| JointGT: Graph-Text Joint Representation Learning for Text Generation from Knowledge Graphs | 1 | 3 | 19 Jun 2021 | not harvested |
| Investigating Pretrained Language Models for Graph-to-Text Generation | 3 | 3 | 16 Jul 2020 | ran 2 of 12 samples (10 unverified) |
Dataset loaders archive 2025-07-28
No loader listed in the archive.
Tasks archive 2025-07-28
License archive 2025-07-28
No licence recorded in the archive. Absence here is not a statement about the dataset's terms.
Modalities archive 2025-07-28
Languages archive 2025-07-28
Variants archive 2025-07-28
- GrailQA
- GrailQA-IID
- GrailQA-Zero-Shot
- GrailQA-Compositional
4 variant names, as the archive lists them.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections