{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/knowhalu-hallucination-detection-via-multi","title":"KnowHalu: Hallucination Detection via Multi-Form Knowledge Based Factual Checking","arxiv_id":"2404.02935","date":"2024-04-03","proceeding":null,"authors":["Jiawei Zhang","Chejian Xu","Yu Gai","Freddy Lecue","Dawn Song","Bo Li"],"abstract":"This paper introduces KnowHalu, a novel approach for detecting hallucinations in text generated by large language models (LLMs), utilizing step-wise reasoning, multi-formulation query, multi-form knowledge for factual checking, and fusion-based detection mechanism. As LLMs are increasingly applied across various domains, ensuring that their outputs are not hallucinated is critical. Recognizing the limitations of existing approaches that either rely on the self-consistency check of LLMs or perform post-hoc fact-checking without considering the complexity of queries or the form of knowledge, KnowHalu proposes a two-phase process for hallucination detection. In the first phase, it identifies non-fabrication hallucinations--responses that, while factually correct, are irrelevant or non-specific to the query. The second phase, multi-form based factual checking, contains five key steps: reasoning and query decomposition, knowledge retrieval, knowledge optimization, judgment generation, and judgment aggregation. Our extensive evaluations demonstrate that KnowHalu significantly outperforms SOTA baselines in detecting hallucinations across diverse tasks, e.g., improving by 15.65% in QA tasks and 5.50% in summarization tasks, highlighting its effectiveness and versatility in detecting hallucinations in LLM-generated content.","url_abs":"https://arxiv.org/abs/2404.02935v1","url_pdf":"https://arxiv.org/pdf/2404.02935v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"knowhalu-hallucination-detection-via-multi","repo_url":"https://github.com/javyduck/knowhalu","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"fact-checking","task_name":"Fact Checking"},{"task_slug":"form","task_name":"Form"},{"task_slug":"hallucination","task_name":"Hallucination"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2404.02935","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.02935"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/javyduck/knowhalu","reach":{"status":"ok"}}],"summary":{"ran":3,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"31ec3bc0276666f0","entry":"clean_query","repo":"javyduck/knowhalu","repo_kind":"official","path":"utils.py","file_url":"https://github.com/javyduck/knowhalu/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"31ec3bc0276666f0"}},{"code_sha256_prefix":"b85a0fb8ce05a717","entry":"extract_query","repo":"javyduck/knowhalu","repo_kind":"official","path":"utils.py","file_url":"https://github.com/javyduck/knowhalu/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b85a0fb8ce05a717"}},{"code_sha256_prefix":"2c8037491021ef54","entry":"insert_newlines","repo":"javyduck/knowhalu","repo_kind":"official","path":"utils.py","file_url":"https://github.com/javyduck/knowhalu/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2c8037491021ef54"}},{"code_sha256_prefix":"9df0d4d165fb9248","entry":"wiki_retrieval","repo":"javyduck/knowhalu","repo_kind":"official","path":"retrieve.py","file_url":"https://github.com/javyduck/knowhalu/blob/HEAD/retrieve.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9df0d4d165fb9248"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}