Papers › Deep Residual Networks with Exponential Linear Unit

Deep Residual Networks with Exponential Linear Unit

14 Apr 2016arXiv:1604.04112archive 2025-07-28

Anish Shah, Eashan Kadam, Hena Shah, Sameer Shinde, Sandip Shingade

Very deep convolutional neural networks introduced new problems like vanishing gradient and degradation. The recent successful contributions towards solving these problems are Residual and Highway Networks. These networks introduce skip connections that allow the information (from the input or those learned in earlier layers) to flow more into the deeper layers. These very deep models have lead to a considerable decrease in test errors, on benchmarks like ImageNet and COCO. In this paper, we propose the use of exponential linear unit instead of the combination of ReLU and Batch Normalization in Residual Networks. We show that this not only speeds up learning in Residual Networks but also improves the accuracy as the depth increases. It improves the test error on almost all data sets, like CIFAR-10 and CIFAR-100

Paper PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

Amihaeseisergiu/Cifar-10-ResNet-ELU-Cutout MIT report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Image Classification

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Image Classification	CIFAR-10	ResNet+ELU	Percentage correct	94.4	#157 of 265	Archive leaderboard	report
Image Classification	CIFAR-100	ResNet+ELU	Percentage correct	73.5	#162 of 211	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

1x1 Convolution Average Pooling Batch Normalization Bottleneck Residual Block Convolution Global Average Pooling Kaiming Initialization Max Pooling ReLU Residual Block Residual Connection

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections