{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-20726","title":"Everywhere Valid Bounds on False Discovery Proportions in Conformal Inference","arxiv_id":"2605.20726","date":"2026-05-20","proceeding":null,"authors":["Ziang Song","Ying Jin","Emmanuel J. Candès"],"abstract":"Modern applications of conformal inference to multiple testing problems, such as outlier detection and candidate selection, often involve selecting test samples whose conformal p-values fall below a threshold. The quality of such methods is often measured by the false discovery proportion (FDP), defined as the fraction of incorrect selections. Existing approaches typically control the expected value of the FDP, using methods such as the Benjamini-Hochberg procedure. This approach fails to provide high-probability bounds on the realized false discovery proportion and invalidates statistical guarantees if the rejection threshold is selected after inspecting the data. This paper establishes finite-sample, distribution-free upper bounds on the FDP that hold simultaneously over all possible rejection thresholds, enabling arbitrary post hoc selection of the threshold. Simultaneous validity is achieved by constructing a high-probability envelope for the empirical distribution function of null conformal p-values by sampling from their joint distribution. Furthermore, our framework allows practitioners to modulate the envelope's shape, thereby producing tight bounds in rejection regions of primary interest. We use this flexible approach to derive simultaneous FDP upper bounds for both outlier detection and conformal selection. We demonstrate through synthetic and real-data experiments that the resulting bounds are both valid and substantially less conservative than those derived from existing approaches.","url_abs":"https://arxiv.org/abs/2605.20726","url_pdf":"https://arxiv.org/pdf/2605.20726","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2605.20726","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.20726"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/sza919/everywhere-valid-fdp-bounds-in-conformal-inference","reach":null}],"summary":{"ran":8,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":10,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"fec8f28b4ae080e6","entry":"DKW_Psi","repo":"sza919/everywhere-valid-fdp-bounds-in-conformal-inference","repo_kind":"found_in_text","path":"cpvals_ecdf.py","file_url":"https://github.com/sza919/everywhere-valid-fdp-bounds-in-conformal-inference/blob/HEAD/cpvals_ecdf.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fec8f28b4ae080e6"}},{"code_sha256_prefix":"eaece170cad45eae","entry":"DKW_lambda","repo":"sza919/everywhere-valid-fdp-bounds-in-conformal-inference","repo_kind":"found_in_text","path":"cpvals_ecdf.py","file_url":"https://github.com/sza919/everywhere-valid-fdp-bounds-in-conformal-inference/blob/HEAD/cpvals_ecdf.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eaece170cad45eae"}},{"code_sha256_prefix":"ffef7c2d15d4407c","entry":"DKW_tau","repo":"sza919/everywhere-valid-fdp-bounds-in-conformal-inference","repo_kind":"found_in_text","path":"cpvals_ecdf.py","file_url":"https://github.com/sza919/everywhere-valid-fdp-bounds-in-conformal-inference/blob/HEAD/cpvals_ecdf.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ffef7c2d15d4407c"}},{"code_sha256_prefix":"3b97ed50a47e706b","entry":"compute_cn","repo":"sza919/everywhere-valid-fdp-bounds-in-conformal-inference","repo_kind":"found_in_text","path":"cccpv/utils_calib.py","file_url":"https://github.com/sza919/everywhere-valid-fdp-bounds-in-conformal-inference/blob/HEAD/cccpv/utils_calib.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3b97ed50a47e706b"}},{"code_sha256_prefix":"6c0e1c48c7271dea","entry":"compute_hybrid_bound","repo":"sza919/everywhere-valid-fdp-bounds-in-conformal-inference","repo_kind":"found_in_text","path":"cccpv/utils_calib.py","file_url":"https://github.com/sza919/everywhere-valid-fdp-bounds-in-conformal-inference/blob/HEAD/cccpv/utils_calib.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6c0e1c48c7271dea"}},{"code_sha256_prefix":"dafab588b576ea20","entry":"estimate_fs_correction","repo":"sza919/everywhere-valid-fdp-bounds-in-conformal-inference","repo_kind":"found_in_text","path":"cccpv/utils_calib.py","file_url":"https://github.com/sza919/everywhere-valid-fdp-bounds-in-conformal-inference/blob/HEAD/cccpv/utils_calib.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dafab588b576ea20"}},{"code_sha256_prefix":"91ca142a4152f58c","entry":"sample_cpvals","repo":"sza919/everywhere-valid-fdp-bounds-in-conformal-inference","repo_kind":"found_in_text","path":"ecdf_upper_bound.py","file_url":"https://github.com/sza919/everywhere-valid-fdp-bounds-in-conformal-inference/blob/HEAD/ecdf_upper_bound.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"91ca142a4152f58c"}},{"code_sha256_prefix":"0be99d75fe80ef67","entry":"uniform_ecdf_upper_bound","repo":"sza919/everywhere-valid-fdp-bounds-in-conformal-inference","repo_kind":"found_in_text","path":"ecdf_upper_bound.py","file_url":"https://github.com/sza919/everywhere-valid-fdp-bounds-in-conformal-inference/blob/HEAD/ecdf_upper_bound.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0be99d75fe80ef67"}},{"code_sha256_prefix":"15104c9b32a56f7e","entry":"calibrate_ccv","repo":"sza919/everywhere-valid-fdp-bounds-in-conformal-inference","repo_kind":"found_in_text","path":"cccpv/methods.py","file_url":"https://github.com/sza919/everywhere-valid-fdp-bounds-in-conformal-inference/blob/HEAD/cccpv/methods.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"15104c9b32a56f7e"}},{"code_sha256_prefix":"628374efbcaf3f10","entry":"uniform_FDP_upper_bound","repo":"sza919/everywhere-valid-fdp-bounds-in-conformal-inference","repo_kind":"found_in_text","path":"ecdf_upper_bound.py","file_url":"https://github.com/sza919/everywhere-valid-fdp-bounds-in-conformal-inference/blob/HEAD/ecdf_upper_bound.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"628374efbcaf3f10"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}