{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/calibrating-reasoning-in-language-models-with","title":"Calibrating Reasoning in Language Models with Internal Consistency","arxiv_id":"2405.18711","date":"2024-05-29","proceeding":null,"authors":["Zhihui Xie","Jizhou Guo","Tong Yu","Shuai Li"],"abstract":"Large language models (LLMs) have demonstrated impressive capabilities in various reasoning tasks, aided by techniques like chain-of-thought prompting that elicits verbalized reasoning. However, LLMs often generate text with obvious mistakes and contradictions, raising doubts about their ability to robustly process and utilize generated rationales. In this work, we investigate reasoning in LLMs through the lens of internal representations, focusing on how these representations are influenced by generated rationales. Our preliminary analysis reveals that while generated rationales improve answer accuracy, inconsistencies emerge between the model's internal representations in middle layers and those in final layers, potentially undermining the reliability of their reasoning processes. To address this, we propose internal consistency as a measure of the model's confidence by examining the agreement of latent predictions decoded from intermediate layers. Extensive empirical studies across different models and datasets demonstrate that internal consistency effectively distinguishes between correct and incorrect reasoning paths. Motivated by this, we propose a new approach to calibrate reasoning by up-weighting reasoning paths with high internal consistency, resulting in a significant boost in reasoning performance. Further analysis uncovers distinct patterns in attention and feed-forward modules across layers, providing insights into the emergence of internal inconsistency. In summary, our results demonstrate the potential of using internal representations for self-evaluation of LLMs. Our code is available at github.com/zhxieml/internal-consistency.","url_abs":"https://arxiv.org/abs/2405.18711v2","url_pdf":"https://arxiv.org/pdf/2405.18711v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"calibrating-reasoning-in-language-models-with","repo_url":"https://github.com/zhxieml/internal-consistency","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.18711","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.18711"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/zhxieml/internal-consistency","reach":null}],"summary":{"ran_draft_wrong":2,"ran_honours":1,"ran_violates":1},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9a0acf728160c813","entry":"cal_agreement_accs","repo":"zhxieml/internal-consistency","repo_kind":"official","path":"src/evaluation/calibrate.py","file_url":"https://github.com/zhxieml/internal-consistency/blob/HEAD/src/evaluation/calibrate.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a0acf728160c813"}},{"code_sha256_prefix":"9a898bf58a992977","entry":"cal_agreement_accs","repo":"zhxieml/internal-consistency","repo_kind":"official","path":"src/evaluation/calibrate.py","file_url":"https://github.com/zhxieml/internal-consistency/blob/HEAD/src/evaluation/calibrate.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a898bf58a992977"}},{"code_sha256_prefix":"7129590c10615e34","entry":"cal_aoc_of_accs","repo":"zhxieml/internal-consistency","repo_kind":"official","path":"src/evaluation/calibrate.py","file_url":"https://github.com/zhxieml/internal-consistency/blob/HEAD/src/evaluation/calibrate.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7129590c10615e34"}},{"code_sha256_prefix":"1d220f96ac433b71","entry":"cal_aoc_of_accs","repo":"zhxieml/internal-consistency","repo_kind":"official","path":"src/evaluation/calibrate.py","file_url":"https://github.com/zhxieml/internal-consistency/blob/HEAD/src/evaluation/calibrate.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1d220f96ac433b71"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}