Methods › General › Neural Architecture Search › DNAS
Differentiable Neural Architecture Search
DNAS
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
DNAS, or Differentiable Neural Architecture Search, uses gradient-based methods to optimize ConvNet architectures, avoiding enumerating and training individual architectures separately as in previous methods. DNAS allows us to explore a layer-wise search space where we can choose a different block for each layer of the network. DNAS represents the search space by a super net whose operators execute stochastically. It relaxes the problem of finding the optimal architecture to find a distribution that yields the optimal architecture. By using the Gumbel Softmax technique, it is possible to directly train the architecture distribution using gradient-based optimization such as SGD.
The loss used to train the stochastic super net consists of both the cross-entropy loss that leads to better accuracy and the latency loss that penalizes the network's latency on a target device. To estimate the latency of an architecture, the latency of each operator in the search space is measured and a lookup table model is used to compute the overall latency by adding up the latency of each operator. Using this model allows for estimation of the latency of architectures in an enormous search space. More importantly, it makes the latency differentiable with respect to layer-wise block choices.
Papers archive 2025-07-28
20 shown of 20, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Learning Novel Transformer Architecture for Time-series Forecasting 19 Feb 2025 · 0 repositories · arXiv:2502.13721
-
Quantum-Enhanced Detection of Viral cDNA via Luminescence Resonance Energy Transfer Using Upconversion and Gold Nanoparticles 14 Oct 2024 · 0 repositories · arXiv:2410.10911
-
Meta-Learning an Evolvable Developmental Encoding 13 Jun 2024 · 1 repository · arXiv:2406.09020
-
Heterogeneous Learning Rate Scheduling for Neural Architecture Search on Long-Tailed Datasets 11 Jun 2024 · 0 repositories · arXiv:2406.07028
-
The devil is in discretization discrepancy. Robustifying Differentiable NAS with Single-Stage Searching Protocol 26 May 2024 · 0 repositories · arXiv:2405.16610
-
MicroNAS: Memory and Latency Constrained Hardware-Aware Neural Architecture Search for Time Series Classification on Microcontrollers 27 Oct 2023 · 0 repositories · arXiv:2310.18384
-
Visual DNA: Representing and Comparing Images using Distributions of Neuron Activations 20 Apr 2023 · 0 repositories · arXiv:2304.10036
-
Efficient Automation of Neural Network Design: A Survey on Differentiable Neural Architecture Search 11 Apr 2023 · 0 repositories · arXiv:2304.05405
-
Tiered Pruning for Efficient Differentialble Inference-Aware Neural Architecture Search 23 Sep 2022 · 0 repositories · arXiv:2209.11785
-
Multi-Complexity-Loss DNAS for Energy-Efficient and Memory-Constrained Deep Neural Networks 1 Jun 2022 · 1 repository · arXiv:2206.00302
-
Approaches to the classification of complex systems: Words, texts, and more 9 May 2022 · 0 repositories · arXiv:2205.04060
-
UDC: Unified DNAS for Compressible TinyML Models 15 Jan 2022 · 0 repositories · arXiv:2201.05842
-
EH-DNAS: End-to-End Hardware-aware Differentiable Neural Architecture Search 24 Nov 2021 · 1 repository · arXiv:2111.12299
-
Differentiable NAS Framework and Application to Ads CTR Prediction 25 Oct 2021 · 1 repository · arXiv:2110.14812
-
RADARS: Memory Efficient Reinforcement Learning Aided Differentiable Neural Architecture Search 13 Sep 2021 · 0 repositories · arXiv:2109.05691
-
Neural Auction: End-to-End Learning of Auction Mechanisms for E-Commerce Advertising 7 Jun 2021 · 0 repositories · arXiv:2106.03593
-
Disentangled Neural Architecture Search 24 Sep 2020 · 0 repositories · arXiv:2009.13266
-
FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions 12 Apr 2020 · 1 repository · arXiv:2004.05565
-
UNAS: Differentiable Architecture Search Meets Reinforcement Learning 16 Dec 2019 · 1 repository · arXiv:1912.07651
-
FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search 9 Dec 2018 · 5 repositories · arXiv:1812.03443Syntology ran 0 of 1 samples · 1 unverified
Tasks archive 2025-07-28
20 shown of 28 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections