{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/improving-uncertainty-estimates-through-the","title":"Improving Calibration through the Relationship with Adversarial Robustness","arxiv_id":"2006.16375","date":"2020-06-29","proceeding":"NeurIPS 2021 12","authors":["Yao Qin","Xuezhi Wang","Alex Beutel","Ed H. Chi"],"abstract":"Neural networks lack adversarial robustness, i.e., they are vulnerable to adversarial examples that through small perturbations to inputs cause incorrect predictions. Further, trust is undermined when models give miscalibrated predictions, i.e., the predicted probability is not a good indicator of how much we should trust our model. In this paper, we study the connection between adversarial robustness and calibration and find that the inputs for which the model is sensitive to small perturbations (are easily attacked) are more likely to have poorly calibrated predictions. Based on this insight, we examine if calibration can be improved by addressing those adversarially unrobust inputs. To this end, we propose Adversarial Robustness based Adaptive Label Smoothing (AR-AdaLS) that integrates the correlations of adversarial robustness and calibration into training by adaptively softening labels for an example based on how easily it can be attacked by an adversary. We find that our method, taking the adversarial robustness of the in-distribution data into consideration, leads to better calibration over the model even under distributional shifts. In addition, AR-AdaLS can also be applied to an ensemble model to further improve model calibration.","url_abs":"https://arxiv.org/abs/2006.16375v2","url_pdf":"https://arxiv.org/pdf/2006.16375v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"adversarial-robustness","task_name":"Adversarial Robustness"}],"methods":[{"method_slug":"label-smoothing","method_name":"Label Smoothing"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2006.16375","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2006.16375"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/tensorflow/cleverhans","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":4},"by_repo_kind":{"found_in_text":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"96b581b6ac010981","entry":"batch_indices","repo":"tensorflow/cleverhans","repo_kind":"found_in_text","path":"cleverhans/utils.py","file_url":"https://github.com/tensorflow/cleverhans/blob/HEAD/cleverhans/utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"96b581b6ac010981"}},{"code_sha256_prefix":"563a92a1c7232061","entry":"other_classes","repo":"tensorflow/cleverhans","repo_kind":"found_in_text","path":"cleverhans/utils.py","file_url":"https://github.com/tensorflow/cleverhans/blob/HEAD/cleverhans/utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"563a92a1c7232061"}},{"code_sha256_prefix":"102904a6c6e09888","entry":"random_feed_dict","repo":"tensorflow/cleverhans","repo_kind":"found_in_text","path":"cleverhans/devtools/mocks.py","file_url":"https://github.com/tensorflow/cleverhans/blob/HEAD/cleverhans/devtools/mocks.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"102904a6c6e09888"}},{"code_sha256_prefix":"de49d6cc85bc9295","entry":"to_categorical","repo":"tensorflow/cleverhans","repo_kind":"found_in_text","path":"cleverhans/utils.py","file_url":"https://github.com/tensorflow/cleverhans/blob/HEAD/cleverhans/utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"de49d6cc85bc9295"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}