{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gradient-masking-causes-clever-to","title":"Gradient Masking Causes CLEVER to Overestimate Adversarial Perturbation Size","arxiv_id":"1804.07870","date":"2018-04-21","proceeding":null,"authors":["Ian Goodfellow"],"abstract":"A key problem in research on adversarial examples is that vulnerability to\nadversarial examples is usually measured by running attack algorithms. Because\nthe attack algorithms are not optimal, the attack algorithms are prone to\noverestimating the size of perturbation needed to fool the target model. In\nother words, the attack-based methodology provides an upper-bound on the size\nof a perturbation that will fool the model, but security guarantees require a\nlower bound. CLEVER is a proposed scoring method to estimate a lower bound.\nUnfortunately, an estimate of a bound is not a bound. In this report, we show\nthat gradient masking, a common problem that causes attack methodologies to\nprovide only a very loose upper bound, causes CLEVER to overestimate the size\nof perturbation needed to fool the model. In other words, CLEVER does not\nresolve the key problem with the attack-based methodology, because it fails to\nprovide a lower bound.","url_abs":"http://arxiv.org/abs/1804.07870v1","url_pdf":"http://arxiv.org/pdf/1804.07870v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"gradient-masking-causes-clever-to","repo_url":"https://github.com/IBM/CLEVER-Robustness-Score","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"gradient-masking-causes-clever-to","repo_url":"https://github.com/huanzhang12/CLEVER","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1804.07870","atlas_url":"https://app.syntology.ai/?focus=1804.07870","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}