Papers › iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients
iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients
Miao Zhang, Steven Su, Shirui Pan, Xiaojun Chang, Ehsan Abbasnejad, Reza Haffari
\textit{Differentiable ARchiTecture Search} (DARTS) has recently become the mainstream of neural architecture search (NAS) due to its efficiency and simplicity. With a gradient-based bi-level optimization, DARTS alternately optimizes the inner model weights and the outer architecture parameter in a weight-sharing supernet. A key challenge to the scalability and quality of the learned architectures is the need for differentiating through the inner-loop optimisation. While much has been discussed about several potentially fatal factors in DARTS, the architecture gradient, a.k.a. hypergradient, has received less attention. In this paper, we tackle the hypergradient computation in DARTS based on the implicit function theorem, making it only depends on the obtained solution to the inner-loop optimization and agnostic to the optimization path. To further reduce the computational requirements, we formulate a stochastic hypergradient approximation for differentiable NAS, and theoretically show that the architecture optimization with the proposed method, named iDARTS, is expected to converge to a stationary point. Comprehensive experiments on two NAS benchmark search spaces and the common NAS search space verify the effectiveness of our proposed method. It leads to architectures outperforming, with large margins, those learned by the baseline methods.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Neural Architecture Search | NAS-Bench-201, CIFAR-10 | iDARTS | Accuracy (Test) | 93.58 | #23 of 37 | Archive leaderboard | report |
| Neural Architecture Search | NAS-Bench-201, CIFAR-10 | iDARTS | Accuracy (Val) | 89.86 | #23 of 37 | Archive leaderboard | report |
| Neural Architecture Search | NAS-Bench-201, CIFAR-100 | iDARTS | Accuracy (Test) | 70.83 | #25 of 40 | Archive leaderboard | report |
| Neural Architecture Search | NAS-Bench-201, CIFAR-100 | iDARTS | Accuracy (Val) | 70.57 | #25 of 40 | Archive leaderboard | report |
| Neural Architecture Search | NAS-Bench-201, ImageNet-16-120 | iDARTS | Accuracy (Test) | 40.89 | #37 of 49 | Archive leaderboard | report |
| Neural Architecture Search | NAS-Bench-201, ImageNet-16-120 | iDARTS | Accuracy (Val) | 40.38 | #37 of 49 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections