Papers › Efficient Architecture Search by Network Transformation

Efficient Architecture Search by Network Transformation

16 Jul 2017arXiv:1707.04873archive 2025-07-28

Han Cai, Tianyao Chen, Wei-Nan Zhang, Yong Yu, Jun Wang

Techniques for automatically designing deep neural network architectures such as reinforcement learning based approaches have recently shown promising results. However, their success is based on vast computational resources (e.g. hundreds of GPUs), making them difficult to be widely used. A noticeable limitation is that they still design and train each network from scratch during the exploration of the architecture space, which is highly inefficient. In this paper, we propose a new framework toward efficient architecture search by exploring the architecture space based on the current network and reusing its weights. We employ a reinforcement learning agent as the meta-controller, whose action is to grow the network depth or layer width with function-preserving transformations. As such, the previously validated networks can be reused for further exploration, thus saves a large amount of computational cost. We apply our method to explore the architecture space of the plain convolutional neural networks (no skip-connections, branching etc.) on image benchmark datasets (CIFAR-10, SVHN) with restricted computational resources (5 GPUs). Our method can design highly competitive networks that outperform existing networks using the same design scheme. On CIFAR-10, our model without skip-connections achieves 4.23\% test error rate, exceeding a vast majority of modern architectures and approaching DenseNet. Furthermore, by applying our method to explore the DenseNet architecture space, we are able to achieve more accurate networks with fewer parameters.

Paper PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

han-cai/EAS officialmentioned in papermentioned on GitHubtfMIT report

han-cai/RL4AS_NetTrans officialmentioned in papermentioned on GitHubtfMIT report

han-cai/PathLevel-EAS mentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Image Classification Neural Architecture Search Reinforcement Learning Reinforcement Learning (RL)reinforcement-learning

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Image Classification	CIFAR-10	RL+NT	Percentage correct	94.6	#154 of 265	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

1x1 Convolution Average Pooling Batch Normalization Concatenated Skip Connection Convolution Dense Block Dense Connections Dropout Global Average Pooling Kaiming Initialization Max Pooling ReLU Softmax

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections