{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/non-exchangeable-conformal-risk-control","title":"Non-Exchangeable Conformal Risk Control","arxiv_id":"2310.01262","date":"2023-10-02","proceeding":null,"authors":["António Farinhas","Chrysoula Zerva","Dennis Ulmer","André F. T. Martins"],"abstract":"Split conformal prediction has recently sparked great interest due to its ability to provide formally guaranteed uncertainty sets or intervals for predictions made by black-box neural models, ensuring a predefined probability of containing the actual ground truth. While the original formulation assumes data exchangeability, some extensions handle non-exchangeable data, which is often the case in many real-world scenarios. In parallel, some progress has been made in conformal methods that provide statistical guarantees for a broader range of objectives, such as bounding the best $F_1$-score or minimizing the false negative rate in expectation. In this paper, we leverage and extend these two lines of work by proposing non-exchangeable conformal risk control, which allows controlling the expected value of any monotone loss function when the data is not exchangeable. Our framework is flexible, makes very few assumptions, and allows weighting the data based on its relevance for a given test example; a careful choice of weights may result on tighter bounds, making our framework useful in the presence of change points, time series, or other forms of distribution drift. Experiments with both synthetic and real world data show the usefulness of our method.","url_abs":"https://arxiv.org/abs/2310.01262v2","url_pdf":"https://arxiv.org/pdf/2310.01262v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"non-exchangeable-conformal-risk-control","repo_url":"https://github.com/deep-spin/non-exchangeable-crc","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"conformal-prediction","task_name":"Conformal Prediction"},{"task_slug":"time-series-1","task_name":"Time Series"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.01262","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.01262"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/deep-spin/non-exchangeable-crc","reach":null}],"summary":{"ran_violates":1,"unverified":3},"by_repo_kind":{"official":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"349a3f7f7e747c7a","entry":"get_lhat_nonx","repo":"deep-spin/non-exchangeable-crc","repo_kind":"official","path":"qa/nonx_risk_histogram.py","file_url":"https://github.com/deep-spin/non-exchangeable-crc/blob/HEAD/qa/nonx_risk_histogram.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"349a3f7f7e747c7a"}},{"code_sha256_prefix":"c3cbd77d333f9c01","entry":"get_lambdas","repo":"deep-spin/non-exchangeable-crc","repo_kind":"official","path":"qa/nonx_risk_histogram.py","file_url":"https://github.com/deep-spin/non-exchangeable-crc/blob/HEAD/qa/nonx_risk_histogram.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c3cbd77d333f9c01"}},{"code_sha256_prefix":"74f154a84c8ad5e0","entry":"get_lhat_nonx","repo":"deep-spin/non-exchangeable-crc","repo_kind":"official","path":"qa/nonx_risk_histogram.py","file_url":"https://github.com/deep-spin/non-exchangeable-crc/blob/HEAD/qa/nonx_risk_histogram.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"74f154a84c8ad5e0"}},{"code_sha256_prefix":"3f13b404f4f00d5f","entry":"get_loss_and_size_tables","repo":"deep-spin/non-exchangeable-crc","repo_kind":"official","path":"qa/nonx_risk_histogram.py","file_url":"https://github.com/deep-spin/non-exchangeable-crc/blob/HEAD/qa/nonx_risk_histogram.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3f13b404f4f00d5f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}