{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/expanding-before-inferring-enhancing","title":"Expanding before Inferring: Enhancing Factuality in Large Language Models through Premature Layers Interpolation","arxiv_id":"2506.02973","date":"2025-06-03","proceeding":null,"authors":["Dingwei Chen","Ziqiang Liu","Feiteng Fang","Chak Tou Leong","Shiwen Ni","Ahmadreza Argha","Hamid Alinejad-Rokny","Min Yang","Chengming Li"],"abstract":"Large Language Models (LLMs) demonstrate remarkable capabilities in text understanding and generation. However, their tendency to produce factually inconsistent outputs, commonly referred to as ''hallucinations'', remains a critical challenge. Existing approaches, such as retrieval-based and inference-time correction methods, primarily address this issue at the input or output level, often overlooking the intrinsic information refinement process and the role of premature layers. Meanwhile, alignment- and fine-tuning-based methods are resource-intensive. In this paper, we propose PLI (Premature Layers Interpolation), a novel, training-free, and plug-and-play intervention designed to enhance factuality. PLI mitigates hallucinations by inserting premature layers formed through mathematical interpolation with adjacent layers. Inspired by stable diffusion and sampling steps, PLI extends the depth of information processing and transmission in LLMs, improving factual coherence. Experiments on four publicly available datasets demonstrate that PLI effectively reduces hallucinations while outperforming existing baselines in most cases. Further analysis suggests that the success of layer interpolation is closely linked to LLMs' internal mechanisms. To promote reproducibility, we will release our code and data upon acceptance.","url_abs":"https://arxiv.org/abs/2506.02973v1","url_pdf":"https://arxiv.org/pdf/2506.02973v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2506.02973","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2506.02973"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/CuSO4-Chen/PLI","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"unverified":4},"by_repo_kind":{"found_in_text":{"samples":5,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e9a7befa1d14a877","entry":"load_jsonl","repo":"CuSO4-Chen/PLI","repo_kind":"found_in_text","path":"src/evaluation/gsm8k_eval.py","file_url":"https://github.com/CuSO4-Chen/PLI/blob/HEAD/src/evaluation/gsm8k_eval.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e9a7befa1d14a877"}},{"code_sha256_prefix":"2c2ea637ced966fa","entry":"build_prompt","repo":"CuSO4-Chen/PLI","repo_kind":"found_in_text","path":"src/evaluation/gsm8k_eval.py","file_url":"https://github.com/CuSO4-Chen/PLI/blob/HEAD/src/evaluation/gsm8k_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2c2ea637ced966fa"}},{"code_sha256_prefix":"158b8847881a64b3","entry":"format_best","repo":"CuSO4-Chen/PLI","repo_kind":"found_in_text","path":"src/evaluation/truthfulqa_eval.py","file_url":"https://github.com/CuSO4-Chen/PLI/blob/HEAD/src/evaluation/truthfulqa_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"158b8847881a64b3"}},{"code_sha256_prefix":"2a4c0ecd291e1795","entry":"load_csv","repo":"CuSO4-Chen/PLI","repo_kind":"found_in_text","path":"src/evaluation/truthfulqa_eval.py","file_url":"https://github.com/CuSO4-Chen/PLI/blob/HEAD/src/evaluation/truthfulqa_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2a4c0ecd291e1795"}},{"code_sha256_prefix":"c852bd4d2face0a1","entry":"split_multi_answer","repo":"CuSO4-Chen/PLI","repo_kind":"found_in_text","path":"src/evaluation/truthfulqa_eval.py","file_url":"https://github.com/CuSO4-Chen/PLI/blob/HEAD/src/evaluation/truthfulqa_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c852bd4d2face0a1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}