Papers › BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rule
BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rule
Miao Zhang, Jilin Hu, Steven Su, Shirui Pan, Xiaojun Chang, Bin Yang, Gholamreza Haffari
Differentiable Architecture Search (DARTS) has received massive attention in recent years, mainly because it significantly reduces the computational cost through weight sharing and continuous relaxation. However, more recent works find that existing differentiable NAS techniques struggle to outperform naive baselines, yielding deteriorative architectures as the search proceeds. Rather than directly optimizing the architecture parameters, this paper formulates the neural architecture search as a distribution learning problem through relaxing the architecture weights into Gaussian distributions. By leveraging the natural-gradient variational inference (NGVI), the architecture distribution can be easily optimized based on existing codebases without incurring more memory and computational consumption. We demonstrate how the differentiable NAS benefits from Bayesian principles, enhancing exploration and improving stability. The experimental results on NAS-Bench-201 and NAS-Bench-1shot1 benchmark datasets confirm the significant improvements the proposed framework can make. In addition, instead of simply applying the argmax on the learned parameters, we further leverage the recently-proposed training-free proxies in NAS to select the optimal architecture from a group architectures drawn from the optimized distribution, where we achieve state-of-the-art results on the NAS-Bench-201 and NAS-Bench-1shot1 benchmarks. Our best architecture in the DARTS search space also obtains competitive test errors with 2.37\%, 15.72\%, and 24.2\% on CIFAR-10, CIFAR-100, and ImageNet datasets, respectively.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Neural Architecture Search | NAS-Bench-201, CIFAR-10 | BaLeNAS-TF | Accuracy (Test) | 94.33 | #10 of 37 | Archive leaderboard | report |
| Neural Architecture Search | NAS-Bench-201, CIFAR-10 | BaLeNAS-TF | Accuracy (Val) | 91.52 | #10 of 37 | Archive leaderboard | report |
| Neural Architecture Search | NAS-Bench-201, CIFAR-100 | BaLeNAS-TF | Accuracy (Test) | 72.95 | #13 of 40 | Archive leaderboard | report |
| Neural Architecture Search | NAS-Bench-201, CIFAR-100 | BaLeNAS-TF | Accuracy (Val) | 72.67 | #13 of 40 | Archive leaderboard | report |
| Neural Architecture Search | NAS-Bench-201, ImageNet-16-120 | BaLeNAS-TF | Accuracy (Test) | 46.54 | #6 of 49 | Archive leaderboard | report |
| Neural Architecture Search | NAS-Bench-201, ImageNet-16-120 | BaLeNAS-TF | Accuracy (Val) | 46.14 | #6 of 49 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections