Browse State-of-the-Art › Distractor Generation
Distractor Generation
17 papers with code · 1 benchmark · 2 datasets archive 2025-07-28
Given a passage, a question, and an answer phrase, the goal of distractor generation (DG) is to generate context-related wrong options (i.e., distractor) for multiple-choice questions (MCQ).
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| RACE (1 row) | BDG p.m. | A BERT-based Distractor Generation Scheme with Multi-tasking and... | code | — | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
2 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 41 · with a code link: 17 · where Syntology ran a sample: 3 (2 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (3 of 41 tagged: 2 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 41 of 41, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
8 Oct 2023 2 repositories listed Syntology 7 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 7 harvested samples) · 3 pointer-only (licence)The success of language models has inspired the NLP community to attend to tasks that require implicit and complex reasoning, relying on human-like commonsense mechanisms.
-
30 Jul 2023 2 repositories listedWe also show the gains of our approach 1 in generating high-quality distractors by comparing it with a zero-shot ChatGPT and a few-shot ChatGPT prompted with static examples.
-
18 Feb 2021 2 repositories listedAs a first step towards measuring news informedness at a scale, we study the problem of quiz-style multiple-choice question generation, which may be used to survey users about their knowledge of recent news.
-
8 Sep 2018 2 repositories listedWe investigate the task of distractor generation for multiple choice reading comprehension questions from examinations.
-
22 Feb 2025 1 repository listedAs by their design, LLMs prioritize generating correct answers, the value of highly plausible yet incorrect answers (candidate answers) tends to be overlooked.
-
26 Sep 2024 1 repository listedRecent advancements in Natural Language Processing (NLP) have impacted numerous sub-fields such as natural language generation, natural language inference, question answering, and more.
-
16 Aug 2024 1 repository listedMultiple-choice questions (MCQs) are important in enhancing concept learning and student engagement for educational purposes.
-
27 Jun 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 3 harvested samples) · 3 pointer-only (licence)High-quality distractors are crucial to both the assessment and pedagogical value of multiple-choice questions (MCQs), where manually crafting ones that anticipate knowledge deficiencies or misconceptions among real…
-
2 Apr 2024 1 repository listedMultiple-choice questions (MCQs) are ubiquitous in almost all levels of education since they are easy to administer, grade, and are a reliable format in assessments and practices.
-
15 Mar 2024 1 repository listedManually designing cloze test consumes enormous time and efforts.
-
13 Jan 2024 1 repository listedWe introduce a multi-stage prompting approach (MSP) for the generation of multiple choice questions (MCQs), harnessing the capabilities of GPT models such as text-davinci-003 and GPT-4, renowned for their excellence…
-
12 Oct 2022 1 repository listedThus, our versatile dataset can be used for both question and distractor generation, as well as to explore new challenges such as question format conversion.
-
9 Aug 2021 1 repository listedAn important part when constructing multiple-choice questions (MCQs) for reading comprehension assessment are the distractors, the incorrect but preferably plausible answer options.
-
3 Jun 2021 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples) · 4 pointer-only (licence)In this framework, we further pre-train mBART sequence-to-sequence denoising auto-encoder model with an auxiliary task using monolingual data of three languages.
-
1 Nov 2020 1 repository listedIn this paper, we investigate the following two limitations for the existing distractor generation (DG) methods.
-
A BERT-based Distractor Generation Scheme with Multi-tasking and Negative Answer Training Strategies12 Oct 2020 1 repository listedIn this paper, we investigate the following two limitations for the existing distractor generation (DG) methods.
-
1 Jun 2018 1 repository listedWe investigate how machine learning models, specifically ranking models, can be used to select useful distractors for multiple choice questions.
-
LookAlike: Consistent Distractor Generation in Math MCQs3 May 2025 0 repositories listedLarge language models (LLMs) are increasingly used to generate distractors for multiple-choice questions (MCQs), especially in domains like math education.
-
D-GEN: Automatic Distractor Generation and Evaluation for Reliable Assessment of Generative Model18 Apr 2025 0 repositories listedEvaluating generative models with open-ended generation is challenging due to inconsistencies in response formats.
-
Do LLMs Make Mistakes Like Students? Exploring Natural Alignment between Language Models and Human Error Patterns21 Feb 2025 0 repositories listedLarge Language Models (LLMs) have demonstrated remarkable capabilities in various educational tasks, yet their alignment with human learning patterns, particularly in predicting which incorrect options students are most…
-
The Imitation Game for Educational AI21 Feb 2025 0 repositories listedAs artificial intelligence systems become increasingly prevalent in education, a fundamental challenge emerges: how can we verify if an AI truly understands how students think and reason?
-
Lost in the Passage: Passage-level In-context Learning Does Not Necessarily Need a "Passage"15 Feb 2025 0 repositories listedBy simply incorporating demonstrations into the context, in-context learning (ICL) enables large language models (LLMs) to yield awesome performance on many tasks.
-
Examining Multilingual Embedding Models Cross-Lingually Through LLM-Generated Adversarial Examples12 Feb 2025 0 repositories listedThe evaluation of cross-lingual semantic search capabilities of models is often limited to existing datasets from tasks such as information retrieval and semantic textual similarity.
-
Generating Plausible Distractors for Multiple-Choice Questions via Student Choice Prediction21 Jan 2025 0 repositories listedIn designing multiple-choice questions (MCQs) in education, creating plausible distractors is crucial for identifying students' misconceptions and gaps in knowledge and accurately assessing their understanding.
-
ISSR: Iterative Selection with Self-Review for Vocabulary Test Distractor Generation7 Jan 2025 0 repositories listedVocabulary acquisition is essential to second language learning, as it underpins all core language skills.
-
Enhancing Distractor Generation for Multiple-Choice Questions with Retrieval Augmented Pretraining and Knowledge Graph Integration19 Jun 2024 0 repositories listedIn this paper, we tackle the task of distractor generation (DG) for multiple-choice questions.
-
Unsupervised Distractor Generation via Large Language Model Distilling and Counterfactual Contrastive Decoding3 Jun 2024 0 repositories listedWithin the context of reading comprehension, the task of Distractor Generation (DG) aims to generate several incorrect options to confuse readers.
-
DGRC: An Effective Fine-tuning Framework for Distractor Generation in Chinese Multi-choice Reading Comprehension29 May 2024 0 repositories listedWhen evaluating a learner's knowledge proficiency, the multiple-choice question is an efficient and widely used format in standardized tests.
-
Improving Automated Distractor Generation for Math Multiple-choice Questions with Overgenerate-and-rank19 Apr 2024 0 repositories listedMultiple-choice questions (MCQs) are commonly used across all levels of math education since they can be deployed and graded at a large scale.
-
Assessing biomedical knowledge robustness in large language models by query-efficient sampling attacks16 Feb 2024 0 repositories listedThe increasing depth of parametric domain knowledge in large language models (LLMs) is fueling their rapid deployment in real-world applications.
-
Distractor Generation in Multiple-Choice Tasks: A Survey of Methods, Datasets, and Evaluation2 Feb 2024 0 repositories listed
-
DISTO: Evaluating Textual Distractors for Multi-Choice Questions using Negative Sampling based Approach10 Apr 2023 0 repositories listed
-
Automatic Distractor Generation for Multiple Choice Questions in Standard Tests26 Nov 2020 0 repositories listed
-
Better Distractions: Transformer-based Distractor Generation and Multiple Choice Question Filtering19 Oct 2020 0 repositories listed
-
Knowledge-Driven Distractor Generation for Cloze-style Multiple Choice Questions21 Apr 2020 0 repositories listed
-
Co-Attention Hierarchical Network: Generating Coherent Long Distractors for Reading Comprehension20 Nov 2019 0 repositories listed
-
Good, Better, Best: Textual Distractors Generation for Multiple-Choice Visual Question Answering via Reinforcement Learning21 Oct 2019 0 repositories listed
-
Equipping Educational Applications with Domain Knowledge1 Aug 2019 0 repositories listed
-
Difficulty-aware Distractor Generation for Gap-Fill Items1 Apr 2019 0 repositories listed
-
Distractor Generation for Chinese Fill-in-the-blank Items1 Sep 2017 0 repositories listed
-
Multiple Choice Question Generation Utilizing An Ontology1 Sep 2017 0 repositories listed
Syntology lines on 3 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections