{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/adversarial-spheres","title":"Adversarial Spheres","arxiv_id":"1801.02774","date":"2018-01-09","proceeding":"ICLR 2018 1","authors":["Justin Gilmer","Luke Metz","Fartash Faghri","Samuel S. Schoenholz","Maithra Raghu","Martin Wattenberg","Ian Goodfellow"],"abstract":"State of the art computer vision models have been shown to be vulnerable to\nsmall adversarial perturbations of the input. In other words, most images in\nthe data distribution are both correctly classified by the model and are very\nclose to a visually similar misclassified image. Despite substantial research\ninterest, the cause of the phenomenon is still poorly understood and remains\nunsolved. We hypothesize that this counter intuitive behavior is a naturally\noccurring result of the high dimensional geometry of the data manifold. As a\nfirst step towards exploring this hypothesis, we study a simple synthetic\ndataset of classifying between two concentric high dimensional spheres. For\nthis dataset we show a fundamental tradeoff between the amount of test error\nand the average distance to nearest error. In particular, we prove that any\nmodel which misclassifies a small constant fraction of a sphere will be\nvulnerable to adversarial perturbations of size $O(1/\\sqrt{d})$. Surprisingly,\nwhen we train several different architectures on this dataset, all of their\nerror sets naturally approach this theoretical bound. As a result of the\ntheory, the vulnerability of neural networks to small adversarial perturbations\nis a logical consequence of the amount of test error observed. We hope that our\ntheoretical analysis of this very simple case will point the way forward to\nexplore how the geometry of complex real-world data sets leads to adversarial\nexamples.","url_abs":"http://arxiv.org/abs/1801.02774v3","url_pdf":"http://arxiv.org/pdf/1801.02774v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"adversarial-spheres","repo_url":"https://github.com/averyma/adversarial-spheres","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"adversarial-spheres","repo_url":"https://github.com/xiaozhanguva/Measure-Concentration","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1801.02774","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}