{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2512-12997","title":"Calibrating Uncertainty for Zero-Shot Adversarial CLIP","arxiv_id":"2512.12997","date":"2025-12-15","proceeding":"ICML","authors":["Wenjing Lu","Zerui Tao","Yuning Qiu","Dongping Zhang","Yang Yang","Qibin Zhao"],"abstract":"CLIP delivers strong zero-shot classification but remains highly vulnerable to adversarial attacks. Prior adversarial fine-tuning work primarily matches predicted logits between clean and adversarial examples, which overlooks uncertainty calibration and may degrade the zero-shot generalization. A common expectation in reliable uncertainty estimation is that predictive uncertainty should increase as inputs become more difficult or shift away from the training distribution. However, we frequently observe the opposite in the adversarial setting: perturbations not only degrade accuracy but also suppress uncertainty, leading to severe miscalibration and over-confidence. This reveals a critical reliability gap beyond robustness. To bridge this gap, we propose an adversarial fine-tuning objective for CLIP considering both accuracy and uncertainty. By reparameterizing CLIP outputs as the concentration parameters of a Dirichlet distribution, we propose a unified representation that captures relative semantic structure and confidence magnitude. This enables holistic distribution alignment under perturbations, moving beyond single-logit anchoring and restoring calibrated uncertainty. Experiments across multiple zero-shot benchmarks demonstrate that our method significantly improves uncertainty calibration and achieves competitive adversarial robustness while preserving clean accuracy.","url_abs":"https://arxiv.org/abs/2512.12997","url_pdf":"https://arxiv.org/pdf/2512.12997","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2512.12997","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2512.12997"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/VivienLu/UCAT","reach":{"status":"ok"}}],"summary":{"ran_fixture":1,"ran_draft_wrong":2,"unverified":19},"by_repo_kind":{"found_in_text":{"samples":22,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":22,"samples":[{"code_sha256_prefix":"b0f936d4d6ae3b8c","entry":"accuracy","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"utils.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/utils.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b0f936d4d6ae3b8c"}},{"code_sha256_prefix":"4c34821ae12949e9","entry":"get_loss","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"slip/slip_models.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/slip/slip_models.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4c34821ae12949e9"}},{"code_sha256_prefix":"345ec4da9f331b85","entry":"get_metric_names","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"slip/slip_models.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/slip/slip_models.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"345ec4da9f331b85"}},{"code_sha256_prefix":"6a6111640475b80b","entry":"attack_pgd","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"attacks.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/attacks.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6a6111640475b80b"}},{"code_sha256_prefix":"3a8a760ecfb5b55a","entry":"attack_pgd","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"attacks_multilabel.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/attacks_multilabel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3a8a760ecfb5b55a"}},{"code_sha256_prefix":"2561f9e79b1a7b83","entry":"au_dirichlet","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"slip-ucat.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/slip-ucat.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2561f9e79b1a7b83"}},{"code_sha256_prefix":"a25e1e252d00d120","entry":"build_model","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"replace/model.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/replace/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a25e1e252d00d120"}},{"code_sha256_prefix":"8a93e041134b597a","entry":"clamp","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"attacks.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/attacks.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8a93e041134b597a"}},{"code_sha256_prefix":"405bd24ae83558d9","entry":"clip_img_preprocessing","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"models/model.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/models/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"405bd24ae83558d9"}},{"code_sha256_prefix":"725e3cb076f6048b","entry":"compute_multilabel_metrics","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"main-multilabel.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/main-multilabel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"725e3cb076f6048b"}},{"code_sha256_prefix":"932d44bbf8bd0e3f","entry":"cosine_lr","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"utils.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"932d44bbf8bd0e3f"}},{"code_sha256_prefix":"cad29a7a417c68d3","entry":"create_logits","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"models/model.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/models/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cad29a7a417c68d3"}},{"code_sha256_prefix":"d2d06e2a020d4fc5","entry":"entropy_from_logits","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"slip-ucat.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/slip-ucat.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d2d06e2a020d4fc5"}},{"code_sha256_prefix":"846e76d01ed68531","entry":"eu_dirichlet","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"slip-ucat.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/slip-ucat.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"846e76d01ed68531"}},{"code_sha256_prefix":"162f47c0241fc73c","entry":"fixed_patch","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"models/prompters.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/models/prompters.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"162f47c0241fc73c"}},{"code_sha256_prefix":"81450906b6e6919b","entry":"getCLIP","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"util/model.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/util/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"81450906b6e6919b"}},{"code_sha256_prefix":"638942a0de2d2288","entry":"normalize","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"models/model.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/models/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"638942a0de2d2288"}},{"code_sha256_prefix":"f5a7aedde2544358","entry":"padding","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"models/prompters.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/models/prompters.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f5a7aedde2544358"}},{"code_sha256_prefix":"cd3ffbc5192b9ccf","entry":"random_patch","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"models/prompters.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/models/prompters.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cd3ffbc5192b9ccf"}},{"code_sha256_prefix":"60ab7dcbb5498f88","entry":"refine_classname","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"utils.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"60ab7dcbb5498f88"}},{"code_sha256_prefix":"8126da3092bcfb30","entry":"reshapeTransform14","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"util/model.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/util/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8126da3092bcfb30"}},{"code_sha256_prefix":"2acf843d6df1edf4","entry":"reshapeTransform7","repo":"VivienLu/UCAT","repo_kind":"found_in_text","path":"util/model.py","file_url":"https://github.com/VivienLu/UCAT/blob/HEAD/util/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2acf843d6df1edf4"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CV","source":"arxiv_api"},"syntology_extracted_results":null}