{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2606-08969","title":"CARE: A Conformal Safety Layer for Medical Summarization","arxiv_id":"2606.08969","date":"2026-06-08","proceeding":null,"authors":["Suhana Bedi","Bridget Lin","Anson Y. Zhou","Chloe O. Stanwyck","Jenelle A. Jindal","Sanmi Koyejo","David Stutz","Nigam H. Shah"],"abstract":"Large language models (LLMs) are increasingly used for medical summarization, but their outputs can omit medically important information and introduce unsupported claims. Existing error-detection methods produce heuristic or uncalibrated scores, providing no formal control over missed errors and no principled way to trade off safety against clinician review burden. We introduce Conformal Assessment for Risk Evaluation (CARE), a post-hoc, model-agnostic safety layer that uses conformal risk control to overlay calibrated omission and hallucination flags onto summaries from any LLM without retraining. CARE provides finite-sample, distribution-free guarantees through two controllers: a hallucination controller that bounds the probability of a document containing any unflagged hallucinated sentence, and an omission controller that bounds the expected fraction of important omissions not surfaced for review. Unlike hallucination detection, omissions depend jointly on whether a source sentence is important and whether it is covered by the summary. We show that calibrating only one dimension can violate the target risk bound, while marginal decompositions remain valid but overly conservative. By jointly calibrating over the full $(τ,γ)$ threshold space, CARE preserves formal guarantees while surfacing up to 5$\\times$ fewer sentences than alternative calibrated baselines. Across five medical summarization tasks, CARE satisfies the target risk bound at $α= 0.15$ with 95% confidence across 100 calibration/test resplits, using only ~100 labeled documents per domain. In a preliminary clinician study (75 document reviews), calibrated flags improved omission detection by 28.6 percentage points on average. These results show that sentence-level safety guarantees are feasible for LLM-assisted medical summarization and offer a tunable mechanism for balancing residual risk and review effort.","url_abs":"https://arxiv.org/abs/2606.08969","url_pdf":"https://arxiv.org/pdf/2606.08969","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2606.08969","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2606.08969"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/som-shahlab/CARE","reach":null}],"summary":{"ran":19,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":21,"ran":19,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":21,"samples":[{"code_sha256_prefix":"9c35861dc2a2adfd","entry":"apply_factuality_controller","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/evaluation.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9c35861dc2a2adfd"}},{"code_sha256_prefix":"bb7ba3c2a62853c2","entry":"apply_filters","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/filters.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/filters.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bb7ba3c2a62853c2"}},{"code_sha256_prefix":"4ac9e0960050a45b","entry":"compute_factuality_loss_binary","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/calibration.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/calibration.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4ac9e0960050a45b"}},{"code_sha256_prefix":"d0c6f72170f181fd","entry":"compute_factuality_risk_binary","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/calibration.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/calibration.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d0c6f72170f181fd"}},{"code_sha256_prefix":"a89d584952e7442f","entry":"compute_omission_loss_binary","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/evaluation.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a89d584952e7442f"}},{"code_sha256_prefix":"e8f0d4a9d6f74900","entry":"format_sentences","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/prompts.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/prompts.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e8f0d4a9d6f74900"}},{"code_sha256_prefix":"f6b5fa69141bf082","entry":"get_phase3_thresholds","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/config.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/config.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f6b5fa69141bf082"}},{"code_sha256_prefix":"aa7faf1fdb6de147","entry":"get_phase4_alpha_sweep","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/config.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/config.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aa7faf1fdb6de147"}},{"code_sha256_prefix":"0a2b743178267463","entry":"get_phase4_results","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/config.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/config.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0a2b743178267463"}},{"code_sha256_prefix":"d214fbfa3e8c5e9b","entry":"is_header_line","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/split.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/split.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d214fbfa3e8c5e9b"}},{"code_sha256_prefix":"fc57bed709eafbe2","entry":"is_rate_limit_error","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/oracle.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/oracle.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fc57bed709eafbe2"}},{"code_sha256_prefix":"ad5a2a3ce53e200d","entry":"is_valid_source_sentence","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/filters.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/filters.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ad5a2a3ce53e200d"}},{"code_sha256_prefix":"604960a428c76815","entry":"is_valid_summary_sentence","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/filters.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/filters.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"604960a428c76815"}},{"code_sha256_prefix":"d062537281fbcda4","entry":"load_jsonl","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/utils.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d062537281fbcda4"}},{"code_sha256_prefix":"3cd11c03e13d6c9c","entry":"protect_abbreviations","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/split.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/split.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3cd11c03e13d6c9c"}},{"code_sha256_prefix":"1ac7912816c40a79","entry":"restore_abbreviations","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/split.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/split.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1ac7912816c40a79"}},{"code_sha256_prefix":"a2e6317c9ba0c260","entry":"setup_logging","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/utils.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a2e6317c9ba0c260"}},{"code_sha256_prefix":"636c9ffc1b6bacac","entry":"simple_average","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/scoring.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/scoring.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"636c9ffc1b6bacac"}},{"code_sha256_prefix":"a3d1665e3446fa6c","entry":"suggested_wait_seconds","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/oracle.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/oracle.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a3d1665e3446fa6c"}},{"code_sha256_prefix":"5939f74f3483d1d6","entry":"get_domain_spec","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/prompts.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/prompts.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5939f74f3483d1d6"}},{"code_sha256_prefix":"4406fcb9e7b308d8","entry":"get_summarizer_system_prompt","repo":"som-shahlab/CARE","repo_kind":"found_in_text","path":"care/prompts.py","file_url":"https://github.com/som-shahlab/CARE/blob/HEAD/care/prompts.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4406fcb9e7b308d8"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}