{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2607-18820","title":"CASE: Causal Alignment and Structural Enforcement for Improving Chain-of-Thought Faithfulness","arxiv_id":"2607.18820","date":"2026-07-21","proceeding":null,"authors":["Ziming Wang","Yinghua Yao","Changwu Huang","Ke Tang","Xin Yao"],"abstract":"Chain-of-thought (CoT) reasoning is widely used to improve both the performance and interpretability of large language models (LLMs), yet the generated reasoning may not faithfully support the final answer. We study this problem from a causal perspective, where a faithful CoT process should follow the chain $Z\\rightarrow X\\rightarrow Y$, with $Z$, $X$, and $Y$ denoting the instruction, reasoning chain, and final answer, respectively. In this process, the instruction should affect the answer only through the reasoning chain. However, conventional autoregressive LLMs condition answer generation on both the instruction and the CoT, which still allows a direct instruction-to-answer shortcut. To address this issue, we propose CASE, a framework that combines training-time causal alignment and inference-time structural enforcement. During training, CASE builds counterfactual-CoT, biased-instruction, and empty-instruction datasets, and applies selective-loss fine-tuning to strengthen CoT-to-answer dependence while suppressing instruction shortcuts. During inference, CASE masks direct attention from instruction tokens to answer tokens, preventing the model from bypassing the generated CoT. We provide an information-theoretic analysis showing how these components promote faithful chains. Experiments on three models and four benchmarks show that CASE achieves a 37\\% average per-setting relative improvement in overall CoT faithfulness over the strongest baselines, exhibits stronger cross-dataset faithfulness transfer, and maintains competitive average accuracy. Code is available at https://github.com/oddwang/CASE.","url_abs":"https://arxiv.org/abs/2607.18820","url_pdf":"https://arxiv.org/pdf/2607.18820","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2607.18820","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2607.18820"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/oddwang/CASE","reach":null}],"summary":{"ran":9},"by_repo_kind":{"found_in_text":{"samples":9,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"6e7faad89a8511f7","entry":"extract_answer_value","repo":"oddwang/CASE","repo_kind":"found_in_text","path":"answer_utils.py","file_url":"https://github.com/oddwang/CASE/blob/HEAD/answer_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6e7faad89a8511f7"}},{"code_sha256_prefix":"479fca296d9c806c","entry":"extract_question_reasoning_answer","repo":"oddwang/CASE","repo_kind":"found_in_text","path":"build_cot_answers.py","file_url":"https://github.com/oddwang/CASE/blob/HEAD/build_cot_answers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"479fca296d9c806c"}},{"code_sha256_prefix":"d70a2acd718aa6e2","entry":"get_cde_counterfactual_question_few_shot","repo":"oddwang/CASE","repo_kind":"found_in_text","path":"prompts/cde_prompts.py","file_url":"https://github.com/oddwang/CASE/blob/HEAD/prompts/cde_prompts.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d70a2acd718aa6e2"}},{"code_sha256_prefix":"e7db11db6b3411da","entry":"get_few_shot_examples","repo":"oddwang/CASE","repo_kind":"found_in_text","path":"build_cot_answers.py","file_url":"https://github.com/oddwang/CASE/blob/HEAD/build_cot_answers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e7db11db6b3411da"}},{"code_sha256_prefix":"294ea6807d05ae54","entry":"is_bool_like","repo":"oddwang/CASE","repo_kind":"found_in_text","path":"CASE_train.py","file_url":"https://github.com/oddwang/CASE/blob/HEAD/CASE_train.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"294ea6807d05ae54"}},{"code_sha256_prefix":"cbc015b753a137ce","entry":"normalize_answer_value","repo":"oddwang/CASE","repo_kind":"found_in_text","path":"answer_utils.py","file_url":"https://github.com/oddwang/CASE/blob/HEAD/answer_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cbc015b753a137ce"}},{"code_sha256_prefix":"907bc60e0618420f","entry":"normalize_index_object","repo":"oddwang/CASE","repo_kind":"found_in_text","path":"CASE_train.py","file_url":"https://github.com/oddwang/CASE/blob/HEAD/CASE_train.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"907bc60e0618420f"}},{"code_sha256_prefix":"3ae23b28a8b8c3ee","entry":"normalize_tf_answer","repo":"oddwang/CASE","repo_kind":"found_in_text","path":"answer_utils.py","file_url":"https://github.com/oddwang/CASE/blob/HEAD/answer_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3ae23b28a8b8c3ee"}},{"code_sha256_prefix":"0cd6052efbccd91c","entry":"to_python_list","repo":"oddwang/CASE","repo_kind":"found_in_text","path":"CASE_train.py","file_url":"https://github.com/oddwang/CASE/blob/HEAD/CASE_train.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0cd6052efbccd91c"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}