{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-large-language-models-hallucination-with","title":"On Large Language Models' Hallucination with Regard to Known Facts","arxiv_id":"2403.20009","date":"2024-03-29","proceeding":null,"authors":["Che Jiang","Biqing Qi","Xiangyu Hong","Dayuan Fu","Yang Cheng","Fandong Meng","Mo Yu","BoWen Zhou","Jie zhou"],"abstract":"Large language models are successful in answering factoid questions but are also prone to hallucination. We investigate the phenomenon of LLMs possessing correct answer knowledge yet still hallucinating from the perspective of inference dynamics, an area not previously covered in studies on hallucinations. We are able to conduct this analysis via two key ideas. First, we identify the factual questions that query the same triplet knowledge but result in different answers. The difference between the model behaviors on the correct and incorrect outputs hence suggests the patterns when hallucinations happen. Second, to measure the pattern, we utilize mappings from the residual streams to vocabulary space. We reveal the different dynamics of the output token probabilities along the depths of layers between the correct and hallucinated cases. In hallucinated cases, the output token's information rarely demonstrates abrupt increases and consistent superiority in the later stages of the model. Leveraging the dynamic curve as a feature, we build a classifier capable of accurately detecting hallucinatory predictions with an 88\\% success rate. Our study shed light on understanding the reasons for LLMs' hallucinations on their known facts, and more importantly, on accurately predicting when they are hallucinating.","url_abs":"https://arxiv.org/abs/2403.20009v2","url_pdf":"https://arxiv.org/pdf/2403.20009v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-large-language-models-hallucination-with","repo_url":"https://github.com/dcdsf321/known_fact_hallucination","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":null}],"tasks":[{"task_slug":"hallucination","task_name":"Hallucination"},{"task_slug":null,"task_name":"Triplet"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.20009","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.20009"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dcdsf321/known_fact_hallucination","reach":null}],"summary":{"ran_honours":1,"ran_draft_wrong":2},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6f8caf5f9c40198a","entry":"compute_score","repo":"dcdsf321/known_fact_hallucination","repo_kind":"official","path":"get_model_output_example_opt.py","file_url":"https://github.com/dcdsf321/known_fact_hallucination/blob/HEAD/get_model_output_example_opt.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6f8caf5f9c40198a"}},{"code_sha256_prefix":"c7189a783afded05","entry":"make_prompt","repo":"dcdsf321/known_fact_hallucination","repo_kind":"official","path":"get_model_output_example_opt.py","file_url":"https://github.com/dcdsf321/known_fact_hallucination/blob/HEAD/get_model_output_example_opt.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c7189a783afded05"}},{"code_sha256_prefix":"50cdaf854b0407b3","entry":"normalize","repo":"dcdsf321/known_fact_hallucination","repo_kind":"official","path":"get_model_output_example_opt.py","file_url":"https://github.com/dcdsf321/known_fact_hallucination/blob/HEAD/get_model_output_example_opt.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"50cdaf854b0407b3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}