{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-pitfalls-and-promise-of-conformal","title":"The Pitfalls and Promise of Conformal Inference Under Adversarial Attacks","arxiv_id":"2405.08886","date":"2024-05-14","proceeding":null,"authors":["Ziquan Liu","Yufei Cui","Yan Yan","Yi Xu","Xiangyang Ji","Xue Liu","Antoni B. Chan"],"abstract":"In safety-critical applications such as medical imaging and autonomous driving, where decisions have profound implications for patient health and road safety, it is imperative to maintain both high adversarial robustness to protect against potential adversarial attacks and reliable uncertainty quantification in decision-making. With extensive research focused on enhancing adversarial robustness through various forms of adversarial training (AT), a notable knowledge gap remains concerning the uncertainty inherent in adversarially trained models. To address this gap, this study investigates the uncertainty of deep learning models by examining the performance of conformal prediction (CP) in the context of standard adversarial attacks within the adversarial defense community. It is first unveiled that existing CP methods do not produce informative prediction sets under the commonly used $l_{\\infty}$-norm bounded attack if the model is not adversarially trained, which underpins the importance of adversarial training for CP. Our paper next demonstrates that the prediction set size (PSS) of CP using adversarially trained models with AT variants is often worse than using standard AT, inspiring us to research into CP-efficient AT for improved PSS. We propose to optimize a Beta-weighting loss with an entropy minimization regularizer during AT to improve CP-efficiency, where the Beta-weighting loss is shown to be an upper bound of PSS at the population level by our theoretical analysis. Moreover, our empirical study on four image classification datasets across three popular AT baselines validates the effectiveness of the proposed Uncertainty-Reducing AT (AT-UR).","url_abs":"https://arxiv.org/abs/2405.08886v1","url_pdf":"https://arxiv.org/pdf/2405.08886v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-pitfalls-and-promise-of-conformal","repo_url":"https://github.com/ziquanliu/ICML2024-AT-UR","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"adversarial-defense","task_name":"Adversarial Defense"},{"task_slug":"adversarial-robustness","task_name":"Adversarial Robustness"},{"task_slug":"autonomous-driving","task_name":"Autonomous Driving"},{"task_slug":"conformal-prediction","task_name":"Conformal Prediction"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"uncertainty-quantification","task_name":"Uncertainty Quantification"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2405.08886","atlas_url":"https://app.syntology.ai/?focus=2405.08886","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.08886"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ziquanliu/ICML2024-AT-UR","reach":{"status":"ok"}}],"summary":{"ran":1,"ran_honours":1,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"3d0f19efa4c17d0b","entry":"attack_image_clean","repo":"ziquanliu/ICML2024-AT-UR","repo_kind":"official","path":"Test_Conformal_Prediction/utils.py","file_url":"https://github.com/ziquanliu/ICML2024-AT-UR/blob/HEAD/Test_Conformal_Prediction/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3d0f19efa4c17d0b"}},{"code_sha256_prefix":"ce9cdd0a73c5e072","entry":"sort_sum","repo":"ziquanliu/ICML2024-AT-UR","repo_kind":"official","path":"Test_Conformal_Prediction/utils.py","file_url":"https://github.com/ziquanliu/ICML2024-AT-UR/blob/HEAD/Test_Conformal_Prediction/utils.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ce9cdd0a73c5e072"}},{"code_sha256_prefix":"c236c11710f87b65","entry":"model_dataset_from_store","repo":"ziquanliu/ICML2024-AT-UR","repo_kind":"official","path":"robustness/model_utils.py","file_url":"https://github.com/ziquanliu/ICML2024-AT-UR/blob/HEAD/robustness/model_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c236c11710f87b65"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}