{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/measuring-the-tendency-of-cnns-to-learn","title":"Measuring the tendency of CNNs to Learn Surface Statistical Regularities","arxiv_id":"1711.11561","date":"2017-11-30","proceeding":null,"authors":["Jason Jo","Yoshua Bengio"],"abstract":"Deep CNNs are known to exhibit the following peculiarity: on the one hand\nthey generalize extremely well to a test set, while on the other hand they are\nextremely sensitive to so-called adversarial perturbations. The extreme\nsensitivity of high performance CNNs to adversarial examples casts serious\ndoubt that these networks are learning high level abstractions in the dataset.\nWe are concerned with the following question: How can a deep CNN that does not\nlearn any high level semantics of the dataset manage to generalize so well? The\ngoal of this article is to measure the tendency of CNNs to learn surface\nstatistical regularities of the dataset. To this end, we use Fourier filtering\nto construct datasets which share the exact same high level abstractions but\nexhibit qualitatively different surface statistical regularities. For the SVHN\nand CIFAR-10 datasets, we present two Fourier filtered variants: a low\nfrequency variant and a randomly filtered variant. Each of the Fourier\nfiltering schemes is tuned to preserve the recognizability of the objects. Our\nmain finding is that CNNs exhibit a tendency to latch onto the Fourier image\nstatistics of the training dataset, sometimes exhibiting up to a 28%\ngeneralization gap across the various test sets. Moreover, we observe that\nsignificantly increasing the depth of a network has a very marginal impact on\nclosing the aforementioned generalization gap. Thus we provide quantitative\nevidence supporting the hypothesis that deep CNNs tend to learn surface\nstatistical regularities in the dataset rather than higher-level abstract\nconcepts.","url_abs":"http://arxiv.org/abs/1711.11561v1","url_pdf":"http://arxiv.org/pdf/1711.11561v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"measuring-the-tendency-of-cnns-to-learn","repo_url":"https://github.com/dtak/local-independence-public","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1711.11561","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}