{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deductive-verification-of-chain-of-thought-1","title":"Deductive Verification of Chain-of-Thought Reasoning","arxiv_id":"2306.03872","date":"2023-06-06","proceeding":"NeurIPS 2023 11","authors":["Zhan Ling","Yunhao Fang","Xuanlin Li","Zhiao Huang","Mingu Lee","Roland Memisevic","Hao Su"],"abstract":"Large Language Models (LLMs) significantly benefit from Chain-of-Thought (CoT) prompting in performing various reasoning tasks. While CoT allows models to produce more comprehensive reasoning processes, its emphasis on intermediate reasoning steps can inadvertently introduce hallucinations and accumulated errors, thereby limiting models' ability to solve complex reasoning tasks. Inspired by how humans engage in careful and meticulous deductive logical reasoning processes to solve tasks, we seek to enable language models to perform explicit and rigorous deductive reasoning, and also ensure the trustworthiness of their reasoning process through self-verification. However, directly verifying the validity of an entire deductive reasoning process is challenging, even with advanced models like ChatGPT. In light of this, we propose to decompose a reasoning verification process into a series of step-by-step subprocesses, each only receiving their necessary context and premises. To facilitate this procedure, we propose Natural Program, a natural language-based deductive reasoning format. Our approach enables models to generate precise reasoning steps where subsequent steps are more rigorously grounded on prior steps. It also empowers language models to carry out reasoning self-verification in a step-by-step manner. By integrating this verification process into each deductive reasoning stage, we significantly enhance the rigor and trustfulness of generated reasoning steps. Along this process, we also improve the answer correctness on complex reasoning tasks. Code will be released at https://github.com/lz1oceani/verify_cot.","url_abs":"https://arxiv.org/abs/2306.03872v3","url_pdf":"https://arxiv.org/pdf/2306.03872v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deductive-verification-of-chain-of-thought-1","repo_url":"https://github.com/lz1oceani/verify_cot","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"logical-reasoning","task_name":"Logical Reasoning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2306.03872","atlas_url":"https://app.syntology.ai/?focus=2306.03872","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.03872"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/lz1oceani/verify_cot","reach":null}],"summary":{"ran_draft_wrong":2,"unverified":5},"by_repo_kind":{"official":{"samples":7,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"81eb84266f4e646e","entry":"lstrip_string","repo":"lz1oceani/verify_cot","repo_kind":"official","path":"run_verification.py","file_url":"https://github.com/lz1oceani/verify_cot/blob/HEAD/run_verification.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"81eb84266f4e646e"}},{"code_sha256_prefix":"2ff8ae19d6efa4a1","entry":"parse_results","repo":"lz1oceani/verify_cot","repo_kind":"official","path":"run_verification.py","file_url":"https://github.com/lz1oceani/verify_cot/blob/HEAD/run_verification.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"2ff8ae19d6efa4a1"}},{"code_sha256_prefix":"5d4397ea6017fb63","entry":"parse_steps","repo":"lz1oceani/verify_cot","repo_kind":"official","path":"run_verification.py","file_url":"https://github.com/lz1oceani/verify_cot/blob/HEAD/run_verification.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"5d4397ea6017fb63"}},{"code_sha256_prefix":"1819d2d2bfe512b2","entry":"prompt_fn","repo":"lz1oceani/verify_cot","repo_kind":"official","path":"prompts.py","file_url":"https://github.com/lz1oceani/verify_cot/blob/HEAD/prompts.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"1819d2d2bfe512b2"}},{"code_sha256_prefix":"4b45db936fa3b443","entry":"prompt_fn","repo":"lz1oceani/verify_cot","repo_kind":"official","path":"run_verification.py","file_url":"https://github.com/lz1oceani/verify_cot/blob/HEAD/run_verification.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"4b45db936fa3b443"}},{"code_sha256_prefix":"9e416d18d332aea8","entry":"run_verify_naive","repo":"lz1oceani/verify_cot","repo_kind":"official","path":"run_verification.py","file_url":"https://github.com/lz1oceani/verify_cot/blob/HEAD/run_verification.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"9e416d18d332aea8"}},{"code_sha256_prefix":"30d79903a57a3423","entry":"verify_steps","repo":"lz1oceani/verify_cot","repo_kind":"official","path":"run_verification.py","file_url":"https://github.com/lz1oceani/verify_cot/blob/HEAD/run_verification.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"30d79903a57a3423"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}