Browse State-of-the-Art › Misconceptions
Misconceptions
54 papers with code · 1 benchmark · 1 dataset archive 2025-07-28
Measures whether a model can discern popular misconceptions from the truth.
Example:
input: The daddy longlegs spider is the most venomous spider in the world.
choice: T
choice: F
answer: F
input: Karl Benz is correctly credited with the invention of the first modern automobile.
choice: T
choice: F
answer: T
Source: BIG-bench
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| BIG-bench (2 rows) | Chinchilla-70B (few-shot, k=5) | Training Compute-Optimal Large Language Models | code | Syntology 8 ran (of which 3 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
1 dataset whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 161 · with a code link: 54 · where Syntology ran a sample: 13 (10 with a run with no instrument failure, 3 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (13 of 161 tagged: 10 with a run with no instrument failure, 3 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 161, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
30 Jul 2016 18 repositories listedCommunity detection in networks is one of the most popular topics of modern network science.
-
28 Jun 2021 6 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 4 unverified (of 5 harvested samples) · 1 pointer-only (licence)Bayesian formulations of deep learning have been shown to have compelling theoretical properties and offer practical functional benefits, such as improved predictive uncertainty quantification and model selection.
-
9 Jun 2022 5 repositories listed Syntology official (archive's flag): 6 ran · 9 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 11 harvested samples) · 6 pointer-only (licence)In this work, we measure and improve the factual accuracy of large-scale LMs for open-ended text generation.
-
13 Jun 2023 3 repositories listedThe release of nuPlan marks a new era in vehicle motion planning research, offering the first large-scale real-world dataset and evaluation schemes requiring both precise short-term planning and long-horizon…
-
8 Dec 2021 3 repositories listedLanguage modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.
-
8 Sep 2021 3 repositories listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)We crafted questions that some humans would answer falsely due to a false belief or misconception.
-
14 Apr 2025 2 repositories listedIn this paper, we first reveal via theoretical derivation that the gradient descent process of the CL objective is formally equivalent to graph convolution, which implies that CL objective inherently supports…
-
9 Jun 2023 2 repositories listedLarge language models (LLMs) have become mainstream technology with their versatile use cases and impressive performance.
-
29 Mar 2022 2 repositories listed Syntology 8 ran (of which 3 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified (of 11 harvested samples) · 4 pointer-only (licence)We investigate the optimal model size and number of tokens for training a transformer language model under a given compute budget.
-
8 Jun 2020 2 repositories listed Syntology official (archive's flag): 8 ran · 13 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 3 honoured, 0 violated, 6 with no contract checked; 4 where Syntology's instrument failed) · 7 unverified (of 20 harvested samples) · 3 pointer-only (licence)Fine-tuning pre-trained transformer-based language models such as BERT has become a common practice dominating leaderboards across various NLP benchmarks.
-
12 Jun 2018 2 repositories listedWe investigate the design challenges of constructing effective and efficient neural sequence labeling systems, by reproducing twelve neural sequence labeling models, which include most of the state-of-the-art…
-
27 May 2025 1 repository listedYounger generations are growing up in a world increasingly shaped by intelligent technologies, making early AI literacy crucial for developing the skills to critically understand and navigate them.
-
17 May 2025 1 repository listedIn this work, we explore a complementary application: using LLMs as verifiers to automate the academic verification of scientific manuscripts.
-
2 May 2025 1 repository listedNext, through an automated pipeline, topic-relevant sections of these concept maps are retrieved to serve as a structured context for the LLM to generate questions and distractors that specifically target common…
-
12 Mar 2025 1 repository listedAs Large Language Models (LLMs) are widely deployed in diverse scenarios, the extent to which they could tacitly spread misinformation emerges as a critical safety concern.
-
5 Mar 2025 1 repository listedUnderstanding the structure of neural network loss surfaces, particularly the emergence of low-loss tunnels, is critical for advancing neural network theory and practice.
-
8 Dec 2024 1 repository listedLarge multimodal models (LMMs) have shown remarkable performance in the visual commonsense reasoning (VCR) task, which aims to answer a multiple-choice question based on visual commonsense within an image.
-
24 Sep 2024 1 repository listedRecent advances in large language models (LLMs) have led to the development of artificial intelligence (AI)-powered tutoring chatbots, showing promise in providing broad access to high-quality personalized education.
-
23 Aug 2024 1 repository listedCRKT benefits KT by directly leveraging answer choices--beyond merely identifying correct or incorrect answers--to distinguish responses with different incorrect choices.
-
3 Jul 2024 1 repository listedWe provide a theoretical explanation of the numerical results presented in support of this claim, describing three narrower classes of functions for which function-generated matrices can be approximated within an…
-
1 Jul 2024 1 repository listedWe propose the Malgorithm Identification task, where LLMs are assessed based on their ability to identify corresponding malgorithm given an incorrect answer choice.
-
27 Jun 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 3 harvested samples) · 3 pointer-only (licence)High-quality distractors are crucial to both the assessment and pedagogical value of multiple-choice questions (MCQs), where manually crafting ones that anticipate knowledge deficiencies or misconceptions among real…
-
30 May 2024 1 repository listedIntelligent Tutoring Systems (ITS) enhance personalized learning by predicting student answers to provide immediate and customized instruction.
-
25 Apr 2024 1 repository listedHowever, we find that evaluation practice is often nebulous.
-
2 Apr 2024 1 repository listedMultiple-choice questions (MCQs) are ubiquitous in almost all levels of education since they are easy to administer, grade, and are a reliable format in assessments and practices.
-
11 Mar 2024 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples) · 7 pointer-only (licence)The popular criticism that errors can compound during autoregressive inference, crucially assumes that teacher-forcing has learned an accurate next-token predictor.
-
11 Mar 2024 1 repository listedIn this work, we explore the efficacy of models in accurately embedding entities within MMKGs through two pivotal tasks: Multi-modal Knowledge Graph Completion (MKGC) and Multi-modal Entity Alignment (MMEA).
-
8 Mar 2024 1 repository listedOften, a good explanation for a program's unexpected behavior is a bug in the programmer's code.
-
2 Mar 2024 1 repository listed Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 9 where Syntology's instrument failed) · 4 unverified (of 13 harvested samples) · 13 pointer-only (licence)Second, we propose a framework for feedback generation that optimizes both correctness and alignment using reinforcement learning (RL).
-
6 Feb 2024 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples)Clarify is the first end-to-end system for user model correction.
-
15 Oct 2023 1 repository listed
-
3 Oct 2023 1 repository listed
-
16 Sep 2023 1 repository listed
-
7 Aug 2023 1 repository listed
-
7 Aug 2023 1 repository listed
-
3 Mar 2023 1 repository listed Syntology official (archive's flag): 1 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 2 pointer-only (licence)
-
2 Oct 2022 1 repository listed
-
19 May 2022 1 repository listed
-
3 May 2022 1 repository listed
-
30 Apr 2022 1 repository listed
-
20 Jan 2022 1 repository listed
-
3 Nov 2021 1 repository listed
-
23 Aug 2021 1 repository listed
-
11 Apr 2021 1 repository listed
-
25 Jan 2021 1 repository listed
-
10 Dec 2020 1 repository listed
-
4 Aug 2020 1 repository listed
-
18 Jun 2020 1 repository listed
-
31 Jan 2020 1 repository listed
-
20 Dec 2019 1 repository listed
-
10 Nov 2019 1 repository listed
-
1 Feb 2019 1 repository listed
-
5 Sep 2018 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 6 harvested samples)
-
28 Feb 2018 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 3 harvested samples) · 3 pointer-only (licence)
-
The Oversmoothing Fallacy: A Misguided Narrative in GNN Research5 Jun 2025 0 repositories listed
-
Automated Identification of Logical Errors in Programs: Advancing Scalable Analysis of Student Misconceptions16 May 2025 0 repositories listed
-
Humans can learn to detect AI-generated texts, or at least learn when they can't3 May 2025 0 repositories listed
-
LLM Library Learning Fails: A LEGO-Prover Case Study3 Apr 2025 0 repositories listed
-
What is AI, what is it not, how we use it in physics and how it impacts... you2 Apr 2025 0 repositories listed
-
From Intuition to Understanding: Using AI Peers to Overcome Physics Misconceptions1 Apr 2025 0 repositories listed
-
Clarifying Misconceptions in COVID-19 Vaccine Sentiment and Stance Analysis and Their Implications for Vaccine Hesitancy Mitigation: A Systematic Review23 Mar 2025 0 repositories listed
-
Emergent Abilities in Large Language Models: A Survey28 Feb 2025 0 repositories listed
-
Analyzing Factors Influencing Driver Willingness to Accept Advanced Driver Assistance Systems23 Feb 2025 0 repositories listed
-
The Imitation Game for Educational AI21 Feb 2025 0 repositories listed
-
Retrieval-augmented systems can be dangerous medical communicators18 Feb 2025 0 repositories listed
-
Foundation Models in Computational Pathology: A Review of Challenges, Opportunities, and Impact12 Feb 2025 0 repositories listed
-
22 Jan 2025 0 repositories listed Syntology 5 ran (of which 2 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples) · 5 pointer-only (licence)
-
Generating Plausible Distractors for Multiple-Choice Questions via Student Choice Prediction21 Jan 2025 0 repositories listed
-
Generative AI in Education: From Foundational Insights to the Socratic Playground for Learning12 Jan 2025 0 repositories listed
-
Decoding Knowledge in Large Language Models: A Framework for Categorization and Comprehension2 Jan 2025 0 repositories listed
-
A Graphical Approach to State Variable Selection in Off-policy Learning1 Jan 2025 0 repositories listed
-
Chasing Progress, Not Perfection: Revisiting Strategies for End-to-End LLM Plan Generation14 Dec 2024 0 repositories listed
-
Developer Perspectives on Licensing and Copyright Issues Arising from Generative AI for Software Development16 Nov 2024 0 repositories listed
-
Automatic Generation of Question Hints for Mathematics Problems using Large Language Models in Educational Technology5 Nov 2024 0 repositories listed
-
A Study on Characterization of Near-Field Sub-Regions For Phased-Array Antennas23 Oct 2024 0 repositories listed
-
LLM-based Cognitive Models of Students with Misconceptions16 Oct 2024 0 repositories listed
-
The Future of Learning in the Age of Generative AI: Automated Question Generation and Assessment with Large Language Models12 Oct 2024 0 repositories listed
-
Benchmark Inflation: Revealing LLM Performance Gaps Using Retro-Holdouts11 Oct 2024 0 repositories listed
-
Listening to Patients: A Framework of Detecting and Mitigating Patient Misreport for Medical Dialogue Generation8 Oct 2024 0 repositories listed
-
Contrastive Explanations That Anticipate Human Misconceptions Can Improve Human Decision-Making Skills5 Oct 2024 0 repositories listed
-
A Thematic Framework for Analyzing Large-scale Self-reported Social Media Data on Opioid Use Disorder Treatment Using Buprenorphine Product2 Oct 2024 0 repositories listed
-
Classifier-Free Guidance is a Predictor-Corrector16 Aug 2024 0 repositories listed
-
Problems in AI, their roots in philosophy, and implications for science and society22 Jul 2024 0 repositories listed
-
Neural topology optimization: the good, the bad, and the ugly19 Jul 2024 0 repositories listed
-
Formalising Anti-Discrimination Law in Automated Decision Systems29 Jun 2024 0 repositories listed
-
Refining Skewed Perceptions in Vision-Language Models through Visual Representations22 May 2024 0 repositories listed
-
An Initial Introduction to Cooperative Multi-Agent Reinforcement Learning10 May 2024 0 repositories listed
-
Toward In-Context Teaching: Adapting Examples to Students' Misconceptions7 May 2024 0 repositories listed
-
Common pitfalls to avoid while using multiobjective optimization in machine learning2 May 2024 0 repositories listed
-
Can a Hallucinating Model help in Reducing Human "Hallucination"?1 May 2024 0 repositories listed
-
Math Multiple Choice Question Generation via Human-Large Language Model Collaboration1 May 2024 0 repositories listed
-
Student Data Paradox and Curious Case of Single Student-Tutor Model: Regressive Side Effects of Training LLMs for Personalized Learning23 Apr 2024 0 repositories listed
-
Differential contributions of machine learning and statistical analysis to language and cognitive sciences22 Apr 2024 0 repositories listed
-
Improving Automated Distractor Generation for Math Multiple-choice Questions with Overgenerate-and-rank19 Apr 2024 0 repositories listed
-
Axiomatic modeling of fixed proportion technologies18 Apr 2024 0 repositories listed
-
Breaking Boundaries: A Chronology with Future Directions of Women in Exercise Physiology Research, Centred on Pregnancy12 Apr 2024 0 repositories listed
-
SoK: On Gradient Leakage in Federated Learning8 Apr 2024 0 repositories listed
-
Prompting the E-Brushes: Users as Authors in Generative AI25 Mar 2024 0 repositories listed
-
The Essential Role of Causality in Foundation World Models for Embodied AI6 Feb 2024 0 repositories listed
-
Enhancing Diagnostic Accuracy through Multi-Agent Conversations: Using Large Language Models to Mitigate Cognitive Bias26 Jan 2024 0 repositories listed
Syntology lines on 13 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections