{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-residual-networks-with-exponential","title":"Deep Residual Networks with Exponential Linear Unit","arxiv_id":"1604.04112","date":"2016-04-14","proceeding":null,"authors":["Anish Shah","Eashan Kadam","Hena Shah","Sameer Shinde","Sandip Shingade"],"abstract":"Very deep convolutional neural networks introduced new problems like\nvanishing gradient and degradation. The recent successful contributions towards\nsolving these problems are Residual and Highway Networks. These networks\nintroduce skip connections that allow the information (from the input or those\nlearned in earlier layers) to flow more into the deeper layers. These very deep\nmodels have lead to a considerable decrease in test errors, on benchmarks like\nImageNet and COCO. In this paper, we propose the use of exponential linear unit\ninstead of the combination of ReLU and Batch Normalization in Residual\nNetworks. We show that this not only speeds up learning in Residual Networks\nbut also improves the accuracy as the depth increases. It improves the test\nerror on almost all data sets, like CIFAR-10 and CIFAR-100","url_abs":"http://arxiv.org/abs/1604.04112v4","url_pdf":"http://arxiv.org/pdf/1604.04112v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-residual-networks-with-exponential","repo_url":"https://github.com/Amihaeseisergiu/Cifar-10-ResNet-ELU-Cutout","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-classification-on-cifar-10","task":"Image Classification","dataset":"CIFAR-10","model":"ResNet+ELU","rank_in_archive_order":157,"of":265,"metrics":{"Percentage correct":"94.4"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-cifar-100","task":"Image Classification","dataset":"CIFAR-100","model":"ResNet+ELU","rank_in_archive_order":162,"of":211,"metrics":{"Percentage correct":"73.5"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1604.04112","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}