{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/understanding-measures-of-uncertainty-for","title":"Understanding Measures of Uncertainty for Adversarial Example Detection","arxiv_id":"1803.08533","date":"2018-03-22","proceeding":null,"authors":["Lewis Smith","Yarin Gal"],"abstract":"Measuring uncertainty is a promising technique for detecting adversarial\nexamples, crafted inputs on which the model predicts an incorrect class with\nhigh confidence. But many measures of uncertainty exist, including predictive\nen- tropy and mutual information, each capturing different types of\nuncertainty. We study these measures, and shed light on why mutual information\nseems to be effective at the task of adversarial example detection. We\nhighlight failure modes for MC dropout, a widely used approach for estimating\nuncertainty in deep models. This leads to an improved understanding of the\ndrawbacks of current methods, and a proposal to improve the quality of\nuncertainty estimates using probabilistic model ensembles. We give illustrative\nexperiments using MNIST to demonstrate the intuition underlying the different\nmeasures of uncertainty, as well as experiments on a real world Kaggle dogs vs\ncats classification dataset.","url_abs":"http://arxiv.org/abs/1803.08533v1","url_pdf":"http://arxiv.org/pdf/1803.08533v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"understanding-measures-of-uncertainty-for","repo_url":"https://github.com/lsgos/uncertainty-adversarial-paper","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"classification","task_name":"General Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1803.08533","atlas_url":"https://app.syntology.ai/?focus=1803.08533","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1803.08533"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lsgos/uncertainty-adversarial-paper","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":8},"by_repo_kind":{"official":{"samples":8,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5cd82d77453fe113","entry":"H","repo":"lsgos/uncertainty-adversarial-paper","repo_kind":"official","path":"latent_plots.py","file_url":"https://github.com/lsgos/uncertainty-adversarial-paper/blob/HEAD/latent_plots.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5cd82d77453fe113"}},{"code_sha256_prefix":"7ae0ad52e475b4da","entry":"create_interpolation_dataset","repo":"lsgos/uncertainty-adversarial-paper","repo_kind":"official","path":"interpolation_comparison.py","file_url":"https://github.com/lsgos/uncertainty-adversarial-paper/blob/HEAD/interpolation_comparison.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7ae0ad52e475b4da"}},{"code_sha256_prefix":"30af533f70831f9f","entry":"crop_center_or_reshape","repo":"lsgos/uncertainty-adversarial-paper","repo_kind":"official","path":"src/utilities.py","file_url":"https://github.com/lsgos/uncertainty-adversarial-paper/blob/HEAD/src/utilities.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"30af533f70831f9f"}},{"code_sha256_prefix":"8b1b6b0189468dfd","entry":"define_VAE","repo":"lsgos/uncertainty-adversarial-paper","repo_kind":"official","path":"train_mnist_vae.py","file_url":"https://github.com/lsgos/uncertainty-adversarial-paper/blob/HEAD/train_mnist_vae.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8b1b6b0189468dfd"}},{"code_sha256_prefix":"ff1b724e6343f9ed","entry":"get_uncertainty_samples","repo":"lsgos/uncertainty-adversarial-paper","repo_kind":"official","path":"latent_plots.py","file_url":"https://github.com/lsgos/uncertainty-adversarial-paper/blob/HEAD/latent_plots.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ff1b724e6343f9ed"}},{"code_sha256_prefix":"2d51388b7dd28bbd","entry":"imagenet_deprocess","repo":"lsgos/uncertainty-adversarial-paper","repo_kind":"official","path":"src/utilities.py","file_url":"https://github.com/lsgos/uncertainty-adversarial-paper/blob/HEAD/src/utilities.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2d51388b7dd28bbd"}},{"code_sha256_prefix":"e2c8245dbaa9881e","entry":"make_grid","repo":"lsgos/uncertainty-adversarial-paper","repo_kind":"official","path":"src/utilities.py","file_url":"https://github.com/lsgos/uncertainty-adversarial-paper/blob/HEAD/src/utilities.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e2c8245dbaa9881e"}},{"code_sha256_prefix":"541c8add17d0bd10","entry":"make_random_targets","repo":"lsgos/uncertainty-adversarial-paper","repo_kind":"official","path":"ROC_curves_cats.py","file_url":"https://github.com/lsgos/uncertainty-adversarial-paper/blob/HEAD/ROC_curves_cats.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"541c8add17d0bd10"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}