Hallucination Evaluation

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

3 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

29 shown of 29 papers with code (49 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models
  23 Oct 2023 9 repositories listed Syntology ran 2 of 8 samples · 6 unverified · 1 pointer-only (licence)
  Our comprehensive case studies within HallusionBench shed light on the challenges of hallucination and illusion in LVLMs.
• AutoHallusion: Automatic Generation of Hallucination Benchmarks for Vision-Language Models
  16 Jun 2024 3 repositories listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  This motivates the development of AutoHallusion, the first automated benchmark generation approach that employs several key strategies to create a diverse range of hallucination examples.
• HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models
  19 May 2023 3 repositories listed Syntology ran 3 of 12 samples · 9 unverified
  Large language models (LLMs), such as ChatGPT, are prone to generate hallucinations, i.
• Alleviating Hallucinations of Large Language Models through Induced Hallucinations
  25 Dec 2023 2 repositories listed Syntology ran 2 of 13 samples · 11 unverified
  Experimental results on both discrimination-based and generation-based hallucination evaluation benchmarks, such as TruthfulQA and \textsc{FActScore}, demonstrate that our proposed ICD methods can effectively enhance…
• KnowRL: Exploring Knowledgeable Reinforcement Learning for Factuality
  24 Jun 2025 1 repository listed
  KnowRL guides models to perform fact-based slow thinking by integrating a factuality reward, based on knowledge verification, into the RL training process, helping them recognize their knowledge boundaries.
• MultiHal: Multilingual Dataset for Knowledge-Graph Grounded Evaluation of LLM Hallucinations
  20 May 2025 1 repository listed
  Large Language Models (LLMs) have inherent limitations of faithfulness and factuality, commonly referred to as hallucinations.
• Benchmarking LLM Faithfulness in RAG with Evolving Leaderboards
  7 May 2025 1 repository listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  We introduce an enhanced hallucination leaderboard centered on FaithJudge, alongside our current hallucination leaderboard, enabling more reliable benchmarking of LLMs for hallucinations in RAG.
• Exploring Hallucination of Large Multimodal Models in Video Understanding: Benchmark, Analysis and Mitigation
  25 Mar 2025 1 repository listed
  The hallucination of large multimodal models (LMMs), providing responses that appear correct but are actually incorrect, limits their reliability and applicability.
• Evaluating LLMs' Assessment of Mixed-Context Hallucination Through the Lens of Summarization
  3 Mar 2025 1 repository listed
  With the rapid development of large language models (LLMs), LLM-as-a-judge has emerged as a widely adopted approach for text quality evaluation, including hallucination evaluation.
• TreeCut: A Synthetic Unanswerable Math Word Problem Dataset for LLM Hallucination Evaluation
  19 Feb 2025 1 repository listed Syntology ran 0 of 3 samples · 3 unverified
  Large language models (LLMs) now achieve near-human performance on standard math word problem benchmarks (e.
• DAHL: Domain-specific Automated Hallucination Evaluation of Long-Form Text through a Benchmark Dataset in Biomedicine
  14 Nov 2024 1 repository listed
  We introduce DAHL, a benchmark dataset and automated evaluation system designed to assess hallucination in long-form text generation, specifically within the biomedical domain.
• DDFAV: Remote Sensing Large Vision Language Models Dataset and Evaluation Benchmark
  5 Nov 2024 1 repository listed
  Next, a training instruction set is produced based on some high-quality remote sensing images selected from the proposed dataset.
• Unified Triplet-Level Hallucination Evaluation for Large Vision-Language Models
  30 Oct 2024 1 repository listed
  The core idea of our framework is to conduct hallucination evaluation on (object, relation, object) triplets extracted from LVLMs' responses, and thus, could be easily generalized to different vision-language tasks.
• LongHalQA: Long-Context Hallucination Evaluation for MultiModal Large Language Models
  13 Oct 2024 1 repository listed
  Several benchmarks have been created to gauge the hallucination levels of MLLMs, by either raising discriminative questions about the existence of objects or introducing LLM evaluators to score the generated text from…
• Evaluating Image Hallucination in Text-to-Image Generation with Question-Answering
  19 Sep 2024 1 repository listed
  Our evaluation protocols measure image hallucination by testing if images from existing TTI models can correctly respond to these questions.
• Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Models
  18 Aug 2024 1 repository listed Syntology ran 2 of 3 samples · 1 unverified · 3 pointer-only (licence)
  Hallucination issues continue to affect multimodal large language models (MLLMs), with existing research mainly addressing object-level or attribute-level hallucinations, neglecting the more complex relation…
• Enhancing LLM's Cognition via Structurization
  23 Jul 2024 1 repository listed Syntology ran 5 of 7 samples · 2 unverified
  Besides, we show the feasibility of distilling advanced LLMs' language processing abilities to a smaller yet effective StruXGPT-7B to execute structurization, addressing the practicality of our approach.
• DefAn: Definitive Answer Dataset for LLMs Hallucination Evaluation
  13 Jun 2024 1 repository listed
  Prompt misalignment hallucination ranges from 6% to 95% in the public dataset and 17% to 94% in the hidden counterpart.
• PhD: A ChatGPT-Prompted Visual hallucination Evaluation Dataset
  17 Mar 2024 1 repository listed
  This paper contributes a ChatGPT-Prompted visual hallucination evaluation Dataset (PhD) for objective VHE at a large scale.
• DiaHalu: A Dialogue-level Hallucination Evaluation Benchmark for Large Language Models
  1 Mar 2024 1 repository listed
  Since large language models (LLMs) achieve significant success in recent years, the hallucination issue remains a challenge, numerous benchmarks are proposed to detect the hallucination.
• TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Space
  27 Feb 2024 1 repository listed Syntology ran 2 of 3 samples · 1 unverified · 3 pointer-only (licence)
  In this paper, we propose TruthX, an inference-time intervention method to activate the truthfulness of LLM by identifying and editing the features within LLM's internal representations that govern the truthfulness.
• Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewrites
  4 Dec 2023 1 repository listed
  The fine-grained object attributes and behaviors non-existent in the image may still be generated but not measured by the current evaluation methods.
• UHGEval: Benchmarking the Hallucination of Chinese Large Language Models via Unconstrained Generation
  26 Nov 2023 1 repository listed Syntology ran 3 of 3 samples · 0 unverified
  These techniques encompass the use of directed hallucination induction and strategies that deliberately alter authentic text to produce hallucinations.
• HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data
  22 Nov 2023 1 repository listed Syntology ran 1 of 7 samples · 6 unverified · 7 pointer-only (licence)
  Multi-modal Large Language Models (MLLMs) tuned on machine-generated instruction-following data have demonstrated remarkable performance in various multi-modal understanding and generation tasks.
• Investigating Hallucinations in Pruned Large Language Models for Abstractive Summarization
  15 Nov 2023 1 repository listed
  Despite the remarkable performance of generative large language models (LLMs) on abstractive summarization, they face two significant challenges: their considerable size and tendency to hallucinate.
• AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation
  13 Nov 2023 1 repository listed Syntology ran 0 of 1 samples · 1 unverified
  Despite making significant progress in multi-modal tasks, current Multi-modal Large Language Models (MLLMs) encounter the significant challenge of hallucinations, which may lead to harmful consequences.
• Analyzing and Mitigating Object Hallucination in Large Vision-Language Models
  1 Oct 2023 1 repository listed Syntology ran 7 of 8 samples · 1 unverified · 8 pointer-only (licence)
  Large vision-language models (LVLMs) have shown remarkable abilities in understanding visual information with human languages.
• Evaluation and Analysis of Hallucination in Large Vision-Language Models
  29 Aug 2023 1 repository listed
  In this paper, we propose Hallucination Evaluation based on Large Language Models (HaELM), an LLM-based hallucination evaluation framework.
• MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Models
  17 Aug 2023 1 repository listed Syntology ran 6 of 8 samples · 2 unverified · 8 pointer-only (licence)
  Large language models (LLMs) have achieved remarkable performance in natural language understanding and generation tasks.

Syntology lines on 14 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.