TruthfulQA

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

3 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 37 papers with code (80 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models
  7 Sep 2023 4 repositories listed Syntology ran 5 of 7 samples · 2 unverified · 7 pointer-only (licence)
  Despite their impressive capabilities, large language models (LLMs) are prone to hallucinations, i.
• RLHF Workflow: From Reward Modeling to Online RLHF
  13 May 2024 3 repositories listed Syntology ran 0 of 1 samples · 1 unverified · 1 pointer-only (licence)
  We present the workflow of Online Iterative Reinforcement Learning from Human Feedback (RLHF) in this technical report, which is widely reported to outperform its offline counterpart by a large margin in the recent…
• TruthfulQA: Measuring How Models Mimic Human Falsehoods
  8 Sep 2021 3 repositories listed Syntology ran 2 of 2 samples · 0 unverified
  We crafted questions that some humans would answer falsely due to a false belief or misconception.
• LACIE: Listener-Aware Finetuning for Confidence Calibration in Large Language Models
  31 May 2024 2 repositories listed
  To calibrate both implicit and explicit confidence markers, we introduce a pragmatic, listener-aware finetuning method (LACIE) that models the listener, considering not only whether an answer is right, but whether it…
• Tuning Language Models by Proxy
  16 Jan 2024 2 repositories listed Syntology ran 8 of 10 samples · 2 unverified · 10 pointer-only (licence)
  Despite the general capabilities of large pretrained language models, they consistently benefit from further adaptation to better achieve desired behaviors.
• Alleviating Hallucinations of Large Language Models through Induced Hallucinations
  25 Dec 2023 2 repositories listed Syntology ran 2 of 13 samples · 11 unverified
  Experimental results on both discrimination-based and generation-based hallucination evaluation benchmarks, such as TruthfulQA and \textsc{FActScore}, demonstrate that our proposed ICD methods can effectively enhance…
• Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment
  18 Aug 2023 2 repositories listed
  In this work, we propose a new safety evaluation benchmark RED-EVAL that carries out red-teaming.
• Unsupervised Elicitation of Language Models
  11 Jun 2025 1 repository listed
  Finally, we show that our method can improve the training of frontier LMs: we use our method to train an unsupervised reward model and use reinforcement learning to train a Claude 3.
• Truth Neurons
  18 May 2025 1 repository listed
  In this paper, we propose a method for identifying representations of truthfulness at the neuron level.
• DeLTa: A Decoding Strategy based on Logit Trajectory Prediction Improves Factuality and Reasoning Ability
  4 Mar 2025 1 repository listed
  Large Language Models (LLMs) are increasingly being used in real-world applications.
• Truth Knows No Language: Evaluating Truthfulness Beyond English
  13 Feb 2025 1 repository listed Syntology ran 0 of 1 samples · 1 unverified
  We introduce a professionally translated extension of the TruthfulQA benchmark designed to evaluate truthfulness in Basque, Catalan, Galician, and Spanish.
• Multi-Agent Reinforcement Learning with Focal Diversity Optimization
  6 Feb 2025 1 repository listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  First, we develop an agent-fusion framework for encouraging multiple LLM based agents to collaborate in producing the final inference output for each LLM query.
• CHAIR -- Classifier of Hallucination as Improver
  5 Jan 2025 1 repository listed
  In this work, we introduce CHAIR (Classifier of Hallucination As ImproveR), a supervised framework for detecting hallucinations by analyzing internal logits from each layer of every token.
• (WhyPHI) Fine-Tuning PHI-3 for Multiple-Choice Question Answering: Methodology, Results, and Challenges
  3 Jan 2025 1 repository listed
  Large Language Models (LLMs) have become essential tools across various domains due to their impressive capabilities in understanding and generating human-like text.
• NoVo: Norm Voting off Hallucinations with Attention Heads in Large Language Models
  11 Oct 2024 1 repository listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  NoVo demonstrates exceptional generalization to 20 diverse datasets, with significant gains in over 90\% of them, far exceeding all current representation editing and reading methods.
• A test suite of prompt injection attacks for LLM-based machine translation
  7 Oct 2024 1 repository listed
  LLM-based NLP systems typically work by embedding their input data into prompt templates which contain instructions and/or in-context examples, creating queries which are submitted to a LLM, and then parsing the LLM…
• Integrative Decoding: Improve Factuality via Implicit Self-consistency
  2 Oct 2024 1 repository listed Syntology ran 1 of 3 samples · 2 unverified · 3 pointer-only (licence)
  Self-consistency-based approaches, which involve repeatedly sampling multiple outputs and selecting the most consistent one as the final response, prove to be remarkably effective in improving the factual accuracy of…
• metabench -- A Sparse Benchmark to Measure General Ability in Large Language Models
  4 Jul 2024 1 repository listed Syntology ran 8 of 9 samples · 1 unverified · 9 pointer-only (licence)
  Large Language Models (LLMs) vary in their abilities on a range of tasks.
• VarBench: Robust Language Model Benchmarking Through Dynamic Variable Perturbation
  25 Jun 2024 1 repository listed Syntology ran 6 of 6 samples · 0 unverified · 6 pointer-only (licence)
  As large language models achieve impressive scores on traditional benchmarks, an increasing number of researchers are becoming concerned about benchmark data leakage during pre-training, commonly known as the data…
• Steering Without Side Effects: Improving Post-Deployment Control of Language Models
  21 Jun 2024 1 repository listed Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
  To demonstrate the generality and transferability of our method beyond jailbreaks, we show that our KTS model can be steered to reduce bias towards user-suggested answers on TruthfulQA.
• Enhancing Language Model Factuality via Activation-Based Confidence Calibration and Guided Decoding
  19 Jun 2024 1 repository listed
  Built on top of ActCab, we further propose CoDec, a confidence-guided decoding strategy to elicit truthful answers with high confidence from LMs.
• Machine Unlearning in Large Language Models
  24 May 2024 1 repository listed
  This paper presents a dual-pronged approach to enhance the ethical and safe behavior of large language models (LLMs) by addressing the issues of harmful responses and copyrighted content.
• Instruction Tuning With Loss Over Instructions
  23 May 2024 1 repository listed Syntology ran 12 of 20 samples · 8 unverified · 20 pointer-only (licence)
  We observe that IM is especially beneficial when trained on datasets with lengthy instructions paired with brief outputs, or under the Superficial Alignment Hypothesis (SAH) where a small amount of training examples are…
• When Hindsight is Not 20/20: Testing Limits on Reflective Thinking in Large Language Models
  14 Apr 2024 1 repository listed Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
  In this paper, we set out to clarify these capabilities under a more stringent evaluation setting in which we disallow any kind of external feedback.
• PoLLMgraph: Unraveling Hallucinations in Large Language Models via State Transition Dynamics
  6 Apr 2024 1 repository listed Syntology ran 6 of 7 samples · 1 unverified · 7 pointer-only (licence)
  Despite tremendous advancements in large language models (LLMs) over recent years, a notably urgent challenge for their practical deployment is the phenomenon of hallucination, where the model fabricates facts and…
• Non-Linear Inference Time Intervention: Improving LLM Truthfulness
  27 Mar 2024 1 repository listed
  In this work, we explore LLM's internal representation space to identify attention heads that contain the most truthful and accurate information.
• In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation
  3 Mar 2024 1 repository listed Syntology ran 2 of 4 samples · 2 unverified · 4 pointer-only (licence)
  Large language models (LLMs) frequently hallucinate and produce factual errors, yet our understanding of why they make these errors remains limited.
• TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Space
  27 Feb 2024 1 repository listed Syntology ran 2 of 3 samples · 1 unverified · 3 pointer-only (licence)
  In this paper, we propose TruthX, an inference-time intervention method to activate the truthfulness of LLM by identifying and editing the features within LLM's internal representations that govern the truthfulness.
• SaGE: Evaluating Moral Consistency in Large Language Models
  21 Feb 2024 1 repository listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  To this extent, we construct the Moral Consistency Corpus (MCC), containing 50K moral questions, responses to them by LLMs, and the RoTs that these models followed.
• Truth Forest: Toward Multi-Scale Truthfulness in Large Language Models through Intervention without Tuning
  29 Dec 2023 1 repository listed
  We introduce Truth Forest, a method that enhances truthfulness in LLMs by uncovering hidden truth representations using multi-dimensional orthogonal probes.

Syntology lines on 18 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.