{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2603-27820","title":"Improving Clinical Diagnosis with Counterfactual Multi-Agent Reasoning","arxiv_id":"2603.27820","date":"2026-03-29","proceeding":null,"authors":["Zhiwen You","Xi Chen","Aniket Vashishtha","Simo Du","Gabriel Erion-Barner","Hongyuan Mei","Hao Peng","Yue Guo"],"abstract":"Clinical diagnosis is a complex reasoning process in which clinicians gather evidence, form hypotheses, and test them against alternative explanations. In medical training, this reasoning is explicitly developed through counterfactual questioning--e.g., asking how a diagnosis would change if a key symptom were absent or altered--to strengthen differential diagnosis skills. As large language model (LLM)-based systems are increasingly used for diagnostic support, ensuring the interpretability of their recommendations becomes critical. However, most existing LLM-based diagnostic agents reason over fixed clinical evidence without explicitly testing how individual findings support or weaken competing diagnoses. In this work, we propose a counterfactual multi-agent diagnostic framework inspired by clinician training that makes hypothesis testing explicit and evidence-grounded. Our framework introduces counterfactual case editing to modify clinical findings and evaluate how these changes affect competing diagnoses. We further define the Counterfactual Probability Gap, a method that quantifies how strongly individual findings support a diagnosis by measuring confidence shifts under these edits. These counterfactual signals guide multi-round specialist discussions, enabling agents to challenge unsupported hypotheses, refine differential diagnoses, and produce more interpretable reasoning trajectories. Across three diagnostic benchmarks and seven LLMs, our method consistently improves diagnostic accuracy over prompting and prior multi-agent baselines, with the largest gains observed in complex and ambiguous cases. Human evaluation further indicates that our framework produces more clinically useful, reliable, and coherent reasoning. These results suggest that incorporating counterfactual evidence verification is an important step toward building reliable AI systems for clinical decision support.","url_abs":"https://arxiv.org/abs/2603.27820","url_pdf":"https://arxiv.org/pdf/2603.27820","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2603.27820","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2603.27820"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/FAIRHealth/clinical-counterfactual-reasoning","reach":null}],"summary":{"ran":1,"ran_draft_wrong":2,"ran_honours":3,"ran_violates":1,"unverified":4},"by_repo_kind":{"found_in_text":{"samples":11,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0625bdf06783c50d","entry":"CounterfactualMetrics","repo":"FAIRHealth/clinical-counterfactual-reasoning","repo_kind":"found_in_text","path":"cfdx/counterfactual.py","file_url":"https://github.com/FAIRHealth/clinical-counterfactual-reasoning/blob/HEAD/cfdx/counterfactual.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0625bdf06783c50d"}},{"code_sha256_prefix":"aed7700612c65e79","entry":"clean_diagnosis_label","repo":"FAIRHealth/clinical-counterfactual-reasoning","repo_kind":"found_in_text","path":"cfdx/counterfactual.py","file_url":"https://github.com/FAIRHealth/clinical-counterfactual-reasoning/blob/HEAD/cfdx/counterfactual.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aed7700612c65e79"}},{"code_sha256_prefix":"22fb39a2cc51d80c","entry":"combine_scores","repo":"FAIRHealth/clinical-counterfactual-reasoning","repo_kind":"found_in_text","path":"cfdx/counterfactual.py","file_url":"https://github.com/FAIRHealth/clinical-counterfactual-reasoning/blob/HEAD/cfdx/counterfactual.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"22fb39a2cc51d80c"}},{"code_sha256_prefix":"0c9d11e96ab96054","entry":"diagnose_top1_with_confidence","repo":"FAIRHealth/clinical-counterfactual-reasoning","repo_kind":"found_in_text","path":"cfdx/counterfactual.py","file_url":"https://github.com/FAIRHealth/clinical-counterfactual-reasoning/blob/HEAD/cfdx/counterfactual.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0c9d11e96ab96054"}},{"code_sha256_prefix":"cc39e17b80d015a7","entry":"is_plausible_diagnosis_label","repo":"FAIRHealth/clinical-counterfactual-reasoning","repo_kind":"found_in_text","path":"cfdx/counterfactual.py","file_url":"https://github.com/FAIRHealth/clinical-counterfactual-reasoning/blob/HEAD/cfdx/counterfactual.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cc39e17b80d015a7"}},{"code_sha256_prefix":"e57f30b06a24e111","entry":"semantic_similarity_sbert","repo":"FAIRHealth/clinical-counterfactual-reasoning","repo_kind":"found_in_text","path":"cfdx/counterfactual.py","file_url":"https://github.com/FAIRHealth/clinical-counterfactual-reasoning/blob/HEAD/cfdx/counterfactual.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e57f30b06a24e111"}},{"code_sha256_prefix":"743017716fa13ae9","entry":"sequence_similarity_ratio","repo":"FAIRHealth/clinical-counterfactual-reasoning","repo_kind":"found_in_text","path":"cfdx/counterfactual.py","file_url":"https://github.com/FAIRHealth/clinical-counterfactual-reasoning/blob/HEAD/cfdx/counterfactual.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"743017716fa13ae9"}},{"code_sha256_prefix":"e59286e7a2aa41a9","entry":"_ensure_sbert_model","repo":"FAIRHealth/clinical-counterfactual-reasoning","repo_kind":"found_in_text","path":"cfdx/counterfactual.py","file_url":"https://github.com/FAIRHealth/clinical-counterfactual-reasoning/blob/HEAD/cfdx/counterfactual.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e59286e7a2aa41a9"}},{"code_sha256_prefix":"725404faaa96731c","entry":"compute_label_shift_score","repo":"FAIRHealth/clinical-counterfactual-reasoning","repo_kind":"found_in_text","path":"cfdx/counterfactual.py","file_url":"https://github.com/FAIRHealth/clinical-counterfactual-reasoning/blob/HEAD/cfdx/counterfactual.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"725404faaa96731c"}},{"code_sha256_prefix":"f99c807bff7f6a8b","entry":"compute_sip_score","repo":"FAIRHealth/clinical-counterfactual-reasoning","repo_kind":"found_in_text","path":"cfdx/counterfactual.py","file_url":"https://github.com/FAIRHealth/clinical-counterfactual-reasoning/blob/HEAD/cfdx/counterfactual.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f99c807bff7f6a8b"}},{"code_sha256_prefix":"82cc2f43242008bf","entry":"evaluate_counterfactual_candidate","repo":"FAIRHealth/clinical-counterfactual-reasoning","repo_kind":"found_in_text","path":"cfdx/counterfactual.py","file_url":"https://github.com/FAIRHealth/clinical-counterfactual-reasoning/blob/HEAD/cfdx/counterfactual.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"82cc2f43242008bf"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}