{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/counterfactual-diagnosis","title":"Counterfactual diagnosis","arxiv_id":"1910.06772","date":"2019-10-15","proceeding":null,"authors":["Jonathan G. Richens","Ciaran M. Lee","Saurabh Johri"],"abstract":"Machine learning promises to revolutionize clinical decision making and diagnosis. In medical diagnosis a doctor aims to explain a patient's symptoms by determining the diseases \\emph{causing} them. However, existing diagnostic algorithms are purely associative, identifying diseases that are strongly correlated with a patients symptoms and medical history. We show that this inability to disentangle correlation from causation can result in sub-optimal or dangerous diagnoses. To overcome this, we reformulate diagnosis as a counterfactual inference task and derive new counterfactual diagnostic algorithms. We show that this approach is closer to the diagnostic reasoning of clinicians and significantly improves the accuracy and safety of the resulting diagnoses. We compare our counterfactual algorithm to the standard Bayesian diagnostic algorithm and a cohort of 44 doctors using a test set of clinical vignettes. While the Bayesian algorithm achieves an accuracy comparable to the average doctor, placing in the top 48% of doctors in our cohort, our counterfactual algorithm places in the top 25% of doctors, achieving expert clinical accuracy. This improvement is achieved simply by changing how we query our model, without requiring any additional model improvements. Our results show that counterfactual reasoning is a vital missing ingredient for applying machine learning to medical diagnosis.","url_abs":"https://arxiv.org/abs/1910.06772v3","url_pdf":"https://arxiv.org/pdf/1910.06772v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"counterfactual-diagnosis","repo_url":"https://github.com/babylonhealth/counterfactual-diagnosis","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"machine-learning","task_name":"BIG-bench Machine Learning"},{"task_slug":"counterfactual-inference","task_name":"Counterfactual Inference"},{"task_slug":"counterfactual-reasoning","task_name":"Counterfactual Reasoning"},{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"diagnostic","task_name":"Diagnostic"},{"task_slug":"medical-diagnosis","task_name":"Medical Diagnosis"},{"task_slug":null,"task_name":"counterfactual"}],"methods":[{"method_slug":"test","method_name":"Test"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1910.06772","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1910.06772"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/babylonhealth/counterfactual-diagnosis","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":8},"by_repo_kind":{"official":{"samples":8,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ff64413107909f8c","entry":"disease_ids","repo":"babylonhealth/counterfactual-diagnosis","repo_kind":"official","path":"inference.py","file_url":"https://github.com/babylonhealth/counterfactual-diagnosis/blob/HEAD/inference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ff64413107909f8c"}},{"code_sha256_prefix":"8e1c8c6bb72249aa","entry":"get_doctor_differential","repo":"babylonhealth/counterfactual-diagnosis","repo_kind":"official","path":"helpers.py","file_url":"https://github.com/babylonhealth/counterfactual-diagnosis/blob/HEAD/helpers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8e1c8c6bb72249aa"}},{"code_sha256_prefix":"6f7d117a5495bd08","entry":"get_evidence_from_casecard","repo":"babylonhealth/counterfactual-diagnosis","repo_kind":"official","path":"inference.py","file_url":"https://github.com/babylonhealth/counterfactual-diagnosis/blob/HEAD/inference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6f7d117a5495bd08"}},{"code_sha256_prefix":"dfb7370128ac17f7","entry":"load_from_json","repo":"babylonhealth/counterfactual-diagnosis","repo_kind":"official","path":"utils.py","file_url":"https://github.com/babylonhealth/counterfactual-diagnosis/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dfb7370128ac17f7"}},{"code_sha256_prefix":"295309e250349980","entry":"load_from_pickle","repo":"babylonhealth/counterfactual-diagnosis","repo_kind":"official","path":"utils.py","file_url":"https://github.com/babylonhealth/counterfactual-diagnosis/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"295309e250349980"}},{"code_sha256_prefix":"fe019b26bf3d5e91","entry":"produce_differentials","repo":"babylonhealth/counterfactual-diagnosis","repo_kind":"official","path":"helpers.py","file_url":"https://github.com/babylonhealth/counterfactual-diagnosis/blob/HEAD/helpers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fe019b26bf3d5e91"}},{"code_sha256_prefix":"b4c2e0c2ba912c98","entry":"risk_factor_ids","repo":"babylonhealth/counterfactual-diagnosis","repo_kind":"official","path":"inference.py","file_url":"https://github.com/babylonhealth/counterfactual-diagnosis/blob/HEAD/inference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b4c2e0c2ba912c98"}},{"code_sha256_prefix":"899871c91882927a","entry":"write_to_pickle","repo":"babylonhealth/counterfactual-diagnosis","repo_kind":"official","path":"utils.py","file_url":"https://github.com/babylonhealth/counterfactual-diagnosis/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"899871c91882927a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}