{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/do-deep-convolutional-nets-really-need-to-be","title":"Do Deep Convolutional Nets Really Need to be Deep and Convolutional?","arxiv_id":"1603.05691","date":"2016-03-17","proceeding":null,"authors":["Gregor Urban","Krzysztof J. Geras","Samira Ebrahimi Kahou","Ozlem Aslan","Shengjie Wang","Rich Caruana","Abdel-rahman Mohamed","Matthai Philipose","Matt Richardson"],"abstract":"Yes, they do. This paper provides the first empirical demonstration that deep\nconvolutional models really need to be both deep and convolutional, even when\ntrained with methods such as distillation that allow small or shallow models of\nhigh accuracy to be trained. Although previous research showed that shallow\nfeed-forward nets sometimes can learn the complex functions previously learned\nby deep nets while using the same number of parameters as the deep models they\nmimic, in this paper we demonstrate that the same methods cannot be used to\ntrain accurate models on CIFAR-10 unless the student models contain multiple\nlayers of convolution. Although the student models do not have to be as deep as\nthe teacher model they mimic, the students need multiple convolutional layers\nto learn functions of comparable accuracy as the deep convolutional teacher.","url_abs":"http://arxiv.org/abs/1603.05691v4","url_pdf":"http://arxiv.org/pdf/1603.05691v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"do-deep-convolutional-nets-really-need-to-be","repo_url":"https://github.com/LeonardoGracioS/DD2424_Deep_Learning_Project","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"do-deep-convolutional-nets-really-need-to-be","repo_url":"https://github.com/LeonardoGracioS/deep_convolutional","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1603.05691","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}