{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/verifiably-robust-conformal-prediction","title":"Verifiably Robust Conformal Prediction","arxiv_id":"2405.18942","date":"2024-05-29","proceeding":null,"authors":["Linus Jeary","Tom Kuipers","Mehran Hosseini","Nicola Paoletti"],"abstract":"Conformal Prediction (CP) is a popular uncertainty quantification method that provides distribution-free, statistically valid prediction sets, assuming that training and test data are exchangeable. In such a case, CP's prediction sets are guaranteed to cover the (unknown) true test output with a user-specified probability. Nevertheless, this guarantee is violated when the data is subjected to adversarial attacks, which often result in a significant loss of coverage. Recently, several approaches have been put forward to recover CP guarantees in this setting. These approaches leverage variations of randomised smoothing to produce conservative sets which account for the effect of the adversarial perturbations. They are, however, limited in that they only support $\\ell^2$-bounded perturbations and classification tasks. This paper introduces VRCP (Verifiably Robust Conformal Prediction), a new framework that leverages recent neural network verification methods to recover coverage guarantees under adversarial attacks. Our VRCP method is the first to support perturbations bounded by arbitrary norms including $\\ell^1$, $\\ell^2$, and $\\ell^\\infty$, as well as regression tasks. We evaluate and compare our approach on image classification tasks (CIFAR10, CIFAR100, and TinyImageNet) and regression tasks for deep reinforcement learning environments. In every case, VRCP achieves above nominal coverage and yields significantly more efficient and informative prediction regions than the SotA.","url_abs":"https://arxiv.org/abs/2405.18942v3","url_pdf":"https://arxiv.org/pdf/2405.18942v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"verifiably-robust-conformal-prediction","repo_url":"https://github.com/ddv-lab/Verifiably_Robust_CP","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"conformal-prediction","task_name":"Conformal Prediction"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"prediction","task_name":"Prediction"},{"task_slug":"uncertainty-quantification","task_name":"Uncertainty Quantification"},{"task_slug":"image-classification","task_name":"image-classification"},{"task_slug":"regression-1","task_name":"regression"},{"task_slug":null,"task_name":"valid"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2405.18942","atlas_url":"https://app.syntology.ai/?focus=2405.18942","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.18942"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ddv-lab/Verifiably_Robust_CP","reach":null}],"summary":{"ran_violates":1,"ran_fixture":4,"ran_draft_wrong":1,"ran":2,"unverified":2},"by_repo_kind":{"official":{"samples":10,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"eee0fffccb507d98","entry":"calculate_Bern_bound","repo":"ddv-lab/Verifiably_Robust_CP","repo_kind":"official","path":"VRCP_Classification/VRCP/experiment.py","file_url":"https://github.com/ddv-lab/Verifiably_Robust_CP/blob/HEAD/VRCP_Classification/VRCP/experiment.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eee0fffccb507d98"}},{"code_sha256_prefix":"0f6a278ae3d77fa8","entry":"calibration","repo":"ddv-lab/Verifiably_Robust_CP","repo_kind":"official","path":"VRCP_Classification/VRCP/experiment.py","file_url":"https://github.com/ddv-lab/Verifiably_Robust_CP/blob/HEAD/VRCP_Classification/VRCP/experiment.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0f6a278ae3d77fa8"}},{"code_sha256_prefix":"b19e659788c63b74","entry":"class_probability_score","repo":"ddv-lab/Verifiably_Robust_CP","repo_kind":"official","path":"VRCP_Classification/VRCP/experiment.py","file_url":"https://github.com/ddv-lab/Verifiably_Robust_CP/blob/HEAD/VRCP_Classification/VRCP/experiment.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b19e659788c63b74"}},{"code_sha256_prefix":"03cd6687b1d4c3be","entry":"evaluate_predictions","repo":"ddv-lab/Verifiably_Robust_CP","repo_kind":"official","path":"VRCP_Classification/VRCP/experiment.py","file_url":"https://github.com/ddv-lab/Verifiably_Robust_CP/blob/HEAD/VRCP_Classification/VRCP/experiment.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"03cd6687b1d4c3be"}},{"code_sha256_prefix":"f780342ea9e8fa2e","entry":"get_scores","repo":"ddv-lab/Verifiably_Robust_CP","repo_kind":"official","path":"VRCP_Classification/VRCP/experiment.py","file_url":"https://github.com/ddv-lab/Verifiably_Robust_CP/blob/HEAD/VRCP_Classification/VRCP/experiment.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f780342ea9e8fa2e"}},{"code_sha256_prefix":"5351b0a2e6494ed5","entry":"prediction","repo":"ddv-lab/Verifiably_Robust_CP","repo_kind":"official","path":"VRCP_Classification/VRCP/experiment.py","file_url":"https://github.com/ddv-lab/Verifiably_Robust_CP/blob/HEAD/VRCP_Classification/VRCP/experiment.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5351b0a2e6494ed5"}},{"code_sha256_prefix":"532b67a2aa47c799","entry":"ranking_score","repo":"ddv-lab/Verifiably_Robust_CP","repo_kind":"official","path":"VRCP_Classification/VRCP/experiment.py","file_url":"https://github.com/ddv-lab/Verifiably_Robust_CP/blob/HEAD/VRCP_Classification/VRCP/experiment.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"532b67a2aa47c799"}},{"code_sha256_prefix":"d7ed63bdfb4157ae","entry":"sigmoid_score","repo":"ddv-lab/Verifiably_Robust_CP","repo_kind":"official","path":"VRCP_Classification/VRCP/experiment.py","file_url":"https://github.com/ddv-lab/Verifiably_Robust_CP/blob/HEAD/VRCP_Classification/VRCP/experiment.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d7ed63bdfb4157ae"}},{"code_sha256_prefix":"3925f3d4836f8634","entry":"_build_scores_list","repo":"ddv-lab/Verifiably_Robust_CP","repo_kind":"official","path":"VRCP_Classification/VRCP/experiment.py","file_url":"https://github.com/ddv-lab/Verifiably_Robust_CP/blob/HEAD/VRCP_Classification/VRCP/experiment.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3925f3d4836f8634"}},{"code_sha256_prefix":"9661e07d6afe5eec","entry":"run_experiments","repo":"ddv-lab/Verifiably_Robust_CP","repo_kind":"official","path":"VRCP_Classification/VRCP/experiment.py","file_url":"https://github.com/ddv-lab/Verifiably_Robust_CP/blob/HEAD/VRCP_Classification/VRCP/experiment.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9661e07d6afe5eec"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}