Papers › Neural Architecture Search using Property Guided Synthesis

Neural Architecture Search using Property Guided Synthesis

8 May 2022arXiv:2205.03960archive 2025-07-28

Charles Jin, Phitchaya Mangpo Phothilimthana, Sudip Roy

In the past few years, neural architecture search (NAS) has become an increasingly important tool within the deep learning community. Despite the many recent successes of NAS, however, most existing approaches operate within highly structured design spaces, and hence explore only a small fraction of the full search space of neural architectures while also requiring significant manual effort from domain experts. In this work, we develop techniques that enable efficient NAS in a significantly larger design space. To accomplish this, we propose to perform NAS in an abstract search space of program properties. Our key insights are as follows: (1) the abstract search space is significantly smaller than the original search space, and (2) architectures with similar program properties also have similar performance; thus, we can search more efficiently in the abstract search space. To enable this approach, we also propose a novel efficient synthesis procedure, which accepts a set of promising program properties, and returns a satisfying neural architecture. We implement our approach, αNAS, within an evolutionary framework, where the mutations are guided by the program properties. Starting with a ResNet-34 model, αNAS produces a model with slightly improved accuracy on CIFAR-10 but 96% fewer parameters. On ImageNet, αNAS is able to improve over Vision Transformer (30% fewer FLOPS and parameters), ResNet-50 (23% fewer FLOPS, 14% fewer parameters), and EfficientNet (7% fewer FLOPS and parameters) without any degradation in accuracy.

PaperPDFCode

Code

google-research/google-research officialmentioned in papertf report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Neural Architecture Search

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

1x1 ConvolutionAbsolute Position EncodingsAdamAttentionAverage PoolingBPEBatch NormalizationConvolutionDense ConnectionsDepthwise ConvolutionDepthwise Separable ConvolutionDropoutInverted Residual BlockLabel SmoothingLayer NormalizationLinear LayerMulti-Head AttentionPointwise ConvolutionPosition-Wise Feed-Forward LayerRMSPropReLUResidual ConnectionSigmoid ActivationSoftmaxSqueeze-and-Excitation BlockTransformerVision Transformer

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections