{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/do-deep-nets-really-need-weight-decay-and","title":"Do deep nets really need weight decay and dropout?","arxiv_id":"1802.07042","date":"2018-02-20","proceeding":null,"authors":["Alex Hernández-García","Peter König"],"abstract":"The impressive success of modern deep neural networks on computer vision\ntasks has been achieved through models of very large capacity compared to the\nnumber of available training examples. This overparameterization is often said\nto be controlled with the help of different regularization techniques, mainly\nweight decay and dropout. However, since these techniques reduce the effective\ncapacity of the model, typically even deeper and wider architectures are\nrequired to compensate for the reduced capacity. Therefore, there seems to be a\nwaste of capacity in this practice. In this paper we build upon recent research\nthat suggests that explicit regularization may not be as important as widely\nbelieved and carry out an ablation study that concludes that weight decay and\ndropout may not be necessary for object recognition if enough data augmentation\nis introduced.","url_abs":"http://arxiv.org/abs/1802.07042v3","url_pdf":"http://arxiv.org/pdf/1802.07042v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"do-deep-nets-really-need-weight-decay-and","repo_url":"https://github.com/oliviawl/image_classification_utkface","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"object-recognition","task_name":"Object Recognition"}],"methods":[{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}