Papers › Fantastic Questions and Where to Find Them: FairytaleQA -- An Authentic Dataset for...

Fantastic Questions and Where to Find Them: FairytaleQA -- An Authentic Dataset for Narrative Comprehension

26 Mar 2022arXiv:2203.13947archive 2025-07-28

Ying Xu, Dakuo Wang, Mo Yu, Daniel Ritchie, Bingsheng Yao, Tongshuang Wu, Zheng Zhang, Toby Jia-Jun Li, Nora Bradford, Branda Sun, Tran Bao Hoang, Yisi Sang, Yufang Hou, Xiaojuan Ma, Diyi Yang, Nanyun Peng, Zhou Yu, Mark Warschauer

Question answering (QA) is a fundamental means to facilitate assessment and training of narrative comprehension skills for both machines and young children, yet there is scarcity of high-quality QA datasets carefully designed to serve this purpose. In particular, existing datasets rarely distinguish fine-grained reading skills, such as the understanding of varying narrative elements. Drawing on the reading education research, we introduce FairytaleQA, a dataset focusing on narrative comprehension of kindergarten to eighth-grade students. Generated by educational experts based on an evidence-based theoretical framework, FairytaleQA consists of 10,580 explicit and implicit questions derived from 278 children-friendly stories, covering seven types of narrative elements or relations. Our dataset is valuable in two folds: First, we ran existing QA models on our dataset and confirmed that this annotation helps assess models' fine-grained learning skills. Second, the dataset supports question generation (QG) task in the education domain. Through benchmarking with QG models, we show that the QG model trained on FairytaleQA is capable of asking high-quality and more diverse questions.

PaperPDFCode

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

uci-soe/fairytaleqadata officialmentioned in papermentioned on GitHubApache-2.0 report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

BenchmarkingQuestion AnsweringQuestion GenerationQuestion-Generation

Datasets

Introduced by this paper, per the archive.

FairytaleQA

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Question Answering FairytaleQA BART fine-tuned on FairytaleQA F1 0.536 #1 of 4 Archive leaderboard report
Question Answering FairytaleQA BART fine-tuned on FairytaleQA Rouge-L 0.533 #1 of 4 Archive leaderboard report
Question Answering FairytaleQA BART fine-tuned on NarrativeQA F1 0.492 #2 of 4 Archive leaderboard report
Question Answering FairytaleQA BART fine-tuned on NarrativeQA Rouge-L 0.475 #2 of 4 Archive leaderboard report
Question Answering FairytaleQA BART F1 0.088 #3 of 4 Archive leaderboard report
Question Answering FairytaleQA BART Rouge-L 0.108 #3 of 4 Archive leaderboard report
Question Answering FairytaleQA DistilBERT F1 0.082 #4 of 4 Archive leaderboard report
Question Answering FairytaleQA DistilBERT Rouge-L 0.097 #4 of 4 Archive leaderboard report
Question Generation FairytaleQA BART fine-tuned on FairytaleQA ROUGE-L 0.527 #1 of 3 Archive leaderboard report
Question Generation FairytaleQA BART fine-tuned on NarrativeQA and FairytaleQA ROUGE-L 0.519 #2 of 3 Archive leaderboard report
Question Generation FairytaleQA BART fine-tuned on NarrativeQA ROUGE-L 0.442 #3 of 3 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections