{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/conformal-risk-minimization-with-variance","title":"Conformal Risk Minimization with Variance Reduction","arxiv_id":"2411.01696","date":"2024-11-03","proceeding":null,"authors":["Sima Noorani","Orlando Romero","Nicolo Dal Fabbro","Hamed Hassani","George J. Pappas"],"abstract":"Conformal prediction (CP) is a distribution-free framework for achieving probabilistic guarantees on black-box models. CP is generally applied to a model post-training. Recent research efforts, on the other hand, have focused on optimizing CP efficiency during training. We formalize this concept as the problem of conformal risk minimization (CRM). In this direction, conformal training (ConfTr) by Stutz et al.(2022) is a technique that seeks to minimize the expected prediction set size of a model by simulating CP in-between training updates. Despite its potential, we identify a strong source of sample inefficiency in ConfTr that leads to overly noisy estimated gradients, introducing training instability and limiting practical use. To address this challenge, we propose variance-reduced conformal training (VR-ConfTr), a CRM method that incorporates a variance reduction technique in the gradient estimation of the ConfTr objective function. Through extensive experiments on various benchmark datasets, we demonstrate that VR-ConfTr consistently achieves faster convergence and smaller prediction sets compared to baselines.","url_abs":"https://arxiv.org/abs/2411.01696v1","url_pdf":"https://arxiv.org/pdf/2411.01696v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"conformal-risk-minimization-with-variance","repo_url":"https://github.com/nooranisima/conformal-risk-minimization-w-variance-reduction-code","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"jax","reach":null}],"tasks":[{"task_slug":"conformal-prediction","task_name":"Conformal Prediction"},{"task_slug":"prediction","task_name":"Prediction"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2411.01696","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.01696"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nooranisima/conformal-risk-minimization-w-variance-reduction-code","reach":null}],"summary":{"ran_draft_wrong":4,"ran_honours":2},"by_repo_kind":{"official":{"samples":6,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"4b759407e042e3f8","entry":"average_results","repo":"nooranisima/conformal-risk-minimization-w-variance-reduction-code","repo_kind":"official","path":"experiments/experiment.py","file_url":"https://github.com/nooranisima/conformal-risk-minimization-w-variance-reduction-code/blob/HEAD/experiments/experiment.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4b759407e042e3f8"}},{"code_sha256_prefix":"4db0c4d0d9f1dd47","entry":"compute_accuracy","repo":"nooranisima/conformal-risk-minimization-w-variance-reduction-code","repo_kind":"official","path":"evaluation/evaluation.py","file_url":"https://github.com/nooranisima/conformal-risk-minimization-w-variance-reduction-code/blob/HEAD/evaluation/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4db0c4d0d9f1dd47"}},{"code_sha256_prefix":"392ce9dab28cbe6c","entry":"evaluate_conformal_prediction","repo":"nooranisima/conformal-risk-minimization-w-variance-reduction-code","repo_kind":"official","path":"evaluation/evaluation.py","file_url":"https://github.com/nooranisima/conformal-risk-minimization-w-variance-reduction-code/blob/HEAD/evaluation/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"392ce9dab28cbe6c"}},{"code_sha256_prefix":"97a695dccd890bd2","entry":"tree_scale","repo":"nooranisima/conformal-risk-minimization-w-variance-reduction-code","repo_kind":"official","path":"algorithms/vr_sort.py","file_url":"https://github.com/nooranisima/conformal-risk-minimization-w-variance-reduction-code/blob/HEAD/algorithms/vr_sort.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"97a695dccd890bd2"}},{"code_sha256_prefix":"9f0a1fa2ca481c2b","entry":"tree_sum","repo":"nooranisima/conformal-risk-minimization-w-variance-reduction-code","repo_kind":"official","path":"algorithms/vr_sort.py","file_url":"https://github.com/nooranisima/conformal-risk-minimization-w-variance-reduction-code/blob/HEAD/algorithms/vr_sort.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9f0a1fa2ca481c2b"}},{"code_sha256_prefix":"4ab89e0539d70ab9","entry":"tree_zero","repo":"nooranisima/conformal-risk-minimization-w-variance-reduction-code","repo_kind":"official","path":"algorithms/vr_sort.py","file_url":"https://github.com/nooranisima/conformal-risk-minimization-w-variance-reduction-code/blob/HEAD/algorithms/vr_sort.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4ab89e0539d70ab9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}