Papers › LayerNAS: Neural Architecture Search in Polynomial Complexity

LayerNAS: Neural Architecture Search in Polynomial Complexity

23 Apr 2023arXiv:2304.11517archive 2025-07-28

Yicheng Fan, Dana Alon, Jingyue Shen, Daiyi Peng, Keshav Kumar, Yun Long, Xin Wang, Fotis Iliopoulos, Da-Cheng Juan, Erik Vee

Neural Architecture Search (NAS) has become a popular method for discovering effective model architectures, especially for target hardware. As such, NAS methods that find optimal architectures under constraints are essential. In our paper, we propose LayerNAS to address the challenge of multi-objective NAS by transforming it into a combinatorial optimization problem, which effectively constrains the search complexity to be polynomial. For a model architecture with L layers, we perform layerwise-search for each layer, selecting from a set of search options 𝕊. LayerNAS groups model candidates based on one objective, such as model size or latency, and searches for the optimal model based on another objective, thereby splitting the cost and reward elements of the search. This approach limits the search complexity to O(H ·|𝕊| ·L), where H is a constant set in LayerNAS. Our experiments show that LayerNAS is able to consistently discover superior models across a variety of search spaces in comparison to strong baselines, including search spaces derived from NATS-Bench, MobileNetV2 and MobileNetV3.

PaperPDF

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

No code repository is listed for this paper in the archive or in Syntology's graph.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Combinatorial OptimizationNeural Architecture Search

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Neural Architecture Search ImageNet LayerNAS-600M MACs 627M #45 of 135 Archive leaderboard report
Neural Architecture Search ImageNet LayerNAS-600M Params 9.7M #45 of 135 Archive leaderboard report
Neural Architecture Search ImageNet LayerNAS-600M Top-1 Error Rate 21.4 #45 of 135 Archive leaderboard report
Neural Architecture Search ImageNet LayerNAS-300M MACs 322M #75 of 135 Archive leaderboard report
Neural Architecture Search ImageNet LayerNAS-300M Params 5.2M #75 of 135 Archive leaderboard report
Neural Architecture Search ImageNet LayerNAS-300M Top-1 Error Rate 22.9 #75 of 135 Archive leaderboard report
Neural Architecture Search ImageNet LayerNAS-220M MACs 229M #106 of 135 Archive leaderboard report
Neural Architecture Search ImageNet LayerNAS-220M Params 5.1M #106 of 135 Archive leaderboard report
Neural Architecture Search ImageNet LayerNAS-220M Top-1 Error Rate 24.4 #106 of 135 Archive leaderboard report
Neural Architecture Search ImageNet LayerNAS-60M MACs 61M #129 of 135 Archive leaderboard report
Neural Architecture Search ImageNet LayerNAS-60M Params 3.7M #129 of 135 Archive leaderboard report
Neural Architecture Search ImageNet LayerNAS-60M Top-1 Error Rate 31 #129 of 135 Archive leaderboard report
Neural Architecture Search NAS-Bench-101 LayerNAS Accuracy (%) 94.26% #2 of 5 Archive leaderboard report
Neural Architecture Search NATS-Bench Size, CIFAR-10 LayerNAS Test Accuracy 93.2 #5 of 5 Archive leaderboard report
Neural Architecture Search NATS-Bench Size, CIFAR-10 LayerNAS Validation Accuracy 0.844 #5 of 5 Archive leaderboard report
Neural Architecture Search NATS-Bench Size, CIFAR-100 LayerNAS Test Accuracy 70.64 #5 of 5 Archive leaderboard report
Neural Architecture Search NATS-Bench Size, CIFAR-100 LayerNAS Validation Accuracy 60.67 #5 of 5 Archive leaderboard report
Neural Architecture Search NATS-Bench Size, ImageNet16-120 LayerNAS Test Accuracy 45.37 #4 of 4 Archive leaderboard report
Neural Architecture Search NATS-Bench Size, ImageNet16-120 LayerNAS Validation Accuracy 38.12 #4 of 4 Archive leaderboard report
Neural Architecture Search NATS-Bench Topology, CIFAR-10 LayerNAS Test Accuracy 94.34±0.12 #1 of 11 Archive leaderboard report
Neural Architecture Search NATS-Bench Topology, CIFAR-100 LayerNAS Test Accuracy 73.01±0.63 #1 of 11 Archive leaderboard report
Neural Architecture Search NATS-Bench Topology, ImageNet16-120 LayerNAS Test Accuracy 46.58±0.59 #1 of 11 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

1x1 ConvolutionAverage PoolingBatch NormalizationConvolutionDense ConnectionsDepthwise ConvolutionDepthwise Separable ConvolutionDropoutGlobal Average PoolingHard SwishInverted Residual BlockPointwise ConvolutionReLUReLU6Sigmoid ActivationSqueeze-and-Excitation Block

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections