Papers › SEDONA: Search for Decoupled Neural Networks toward Greedy Block-wise Learning

SEDONA: Search for Decoupled Neural Networks toward Greedy Block-wise Learning

1 Jan 2021ICLR 2021 1archive 2025-07-28

Myeongjang Pyeon, Jihwan Moon, Taeyoung Hahn, Gunhee Kim

Backward locking and update locking are well-known sources of inefficiency in backpropagation that prevent from concurrently updating layers. Several works have recently suggested using local error signals to train network blocks asynchronously to overcome these limitations. However, they often require numerous iterations of trial-and-error to find the best configuration for local training, including how to decouple network blocks and which auxiliary networks to use for each block. In this work, we propose a differentiable search algorithm named SEDONA to automate this process. Experimental results show that our algorithm can consistently discover transferable decoupled architectures for VGG and ResNet variants, and significantly outperforms the ones trained with end-to-end backpropagation and other state-of-the-art greedy-leaning methods in CIFAR-10, Tiny-ImageNet and ImageNet. Thanks to improved parallelism by local training, we also report up to 2.02× speedup over backpropagation in total training time.

Paper PDF Code

Code

mjpyeon/sedona pytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Neural Architecture Search

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Neural Architecture Search	ImageNet	SEDONA (ResNet-152 Aux. Ens., K=4)	Top-1 Error Rate	20.2	#26 of 135	Archive leaderboard	report
Neural Architecture Search	ImageNet	SEDONA (ResNet-152, K=4)	Top-1 Error Rate	21.09	#38 of 135	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

1x1 Convolution Average Pooling Batch Normalization Bottleneck Residual Block Convolution Dense Connections Dropout Global Average Pooling Kaiming Initialization Max Pooling ReLU Residual Block Residual Connection Softmax

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections