{"url":"/method/dnas","slug":"dnas","name":"DNAS","full_name":"Differentiable Neural Architecture Search","full_name_withheld":false,"description_markdown":"**DNAS**, or **Differentiable Neural Architecture Search**, uses gradient-based methods to optimize ConvNet architectures, avoiding enumerating and training individual architectures separately as in previous methods. DNAS allows us to explore a layer-wise search space where we can choose a different block for each layer of the network. DNAS represents the search space by a super net whose operators execute stochastically. It relaxes the problem of finding the optimal architecture to find a distribution that yields the optimal architecture. By using the [Gumbel Softmax](https://paperswithcode.com/method/gumbel-softmax) technique, it is possible to directly train the architecture distribution using gradient-based optimization such as [SGD](https://paperswithcode.com/method/sgd).\r\n\r\nThe loss used to train the stochastic super net consists of both the cross-entropy loss that leads to better accuracy and the latency loss that penalizes the network's latency on a target device. To estimate the latency of an architecture, the latency of each operator in the search space is measured and a lookup table model is used to compute the overall latency by adding up the latency of each operator. Using this model allows for estimation of the latency of architectures in an enormous search space. More importantly, it makes the latency differentiable with respect to layer-wise block choices.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"https://arxiv.org/abs/1812.03443v3","title":"FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Neural Architecture Search","url":"/methods/category/neural-architecture-search","pwc_aliases":[]}],"n_papers_tagged":20,"archive_num_papers":null,"papers_newest_first":[{"paper":null,"title":"Learning Novel Transformer Architecture for Time-series Forecasting","date":"2025-02-19","arxiv_id":"2502.13721","n_code_links":0,"syntology":null},{"paper":null,"title":"Quantum-Enhanced Detection of Viral cDNA via Luminescence Resonance Energy Transfer Using Upconversion and Gold Nanoparticles","date":"2024-10-14","arxiv_id":"2410.10911","n_code_links":0,"syntology":null},{"paper":"/paper/meta-learning-an-evolvable-developmental","title":"Meta-Learning an Evolvable Developmental Encoding","date":"2024-06-13","arxiv_id":"2406.09020","n_code_links":1,"syntology":null},{"paper":null,"title":"Heterogeneous Learning Rate Scheduling for Neural Architecture Search on Long-Tailed Datasets","date":"2024-06-11","arxiv_id":"2406.07028","n_code_links":0,"syntology":null},{"paper":null,"title":"The devil is in discretization discrepancy. Robustifying Differentiable NAS with Single-Stage Searching Protocol","date":"2024-05-26","arxiv_id":"2405.16610","n_code_links":0,"syntology":null},{"paper":null,"title":"MicroNAS: Memory and Latency Constrained Hardware-Aware Neural Architecture Search for Time Series Classification on Microcontrollers","date":"2023-10-27","arxiv_id":"2310.18384","n_code_links":0,"syntology":null},{"paper":null,"title":"Visual DNA: Representing and Comparing Images using Distributions of Neuron Activations","date":"2023-04-20","arxiv_id":"2304.10036","n_code_links":0,"syntology":null},{"paper":null,"title":"Efficient Automation of Neural Network Design: A Survey on Differentiable Neural Architecture Search","date":"2023-04-11","arxiv_id":"2304.05405","n_code_links":0,"syntology":null},{"paper":null,"title":"Tiered Pruning for Efficient Differentialble Inference-Aware Neural Architecture Search","date":"2022-09-23","arxiv_id":"2209.11785","n_code_links":0,"syntology":null},{"paper":"/paper/multi-complexity-loss-dnas-for-energy","title":"Multi-Complexity-Loss DNAS for Energy-Efficient and Memory-Constrained Deep Neural Networks","date":"2022-06-01","arxiv_id":"2206.00302","n_code_links":1,"syntology":null},{"paper":null,"title":"Approaches to the classification of complex systems: Words, texts, and more","date":"2022-05-09","arxiv_id":"2205.04060","n_code_links":0,"syntology":null},{"paper":null,"title":"UDC: Unified DNAS for Compressible TinyML Models","date":"2022-01-15","arxiv_id":"2201.05842","n_code_links":0,"syntology":null},{"paper":"/paper/eh-dnas-end-to-end-hardware-aware","title":"EH-DNAS: End-to-End Hardware-aware Differentiable Neural Architecture Search","date":"2021-11-24","arxiv_id":"2111.12299","n_code_links":1,"syntology":null},{"paper":"/paper/differentiable-nas-framework-and-application","title":"Differentiable NAS Framework and Application to Ads CTR Prediction","date":"2021-10-25","arxiv_id":"2110.14812","n_code_links":1,"syntology":null},{"paper":null,"title":"RADARS: Memory Efficient Reinforcement Learning Aided Differentiable Neural Architecture Search","date":"2021-09-13","arxiv_id":"2109.05691","n_code_links":0,"syntology":null},{"paper":null,"title":"Neural Auction: End-to-End Learning of Auction Mechanisms for E-Commerce Advertising","date":"2021-06-07","arxiv_id":"2106.03593","n_code_links":0,"syntology":null},{"paper":null,"title":"Disentangled Neural Architecture Search","date":"2020-09-24","arxiv_id":"2009.13266","n_code_links":0,"syntology":null},{"paper":"/paper/fbnetv2-differentiable-neural-architecture","title":"FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions","date":"2020-04-12","arxiv_id":"2004.05565","n_code_links":1,"syntology":null},{"paper":"/paper/unas-differentiable-architecture-search-meets","title":"UNAS: Differentiable Architecture Search Meets Reinforcement Learning","date":"2019-12-16","arxiv_id":"1912.07651","n_code_links":1,"syntology":null},{"paper":"/paper/fbnet-hardware-aware-efficient-convnet-design","title":"FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search","date":"2018-12-09","arxiv_id":"1812.03443","n_code_links":5,"syntology":{"ran":0,"of":1,"unverified":1,"pointer_only":0}}],"papers_shown":20,"tasks":[{"task":"/task/architecture-search","name":"Neural Architecture Search","papers":15},{"task":null,"name":"GPU","papers":3},{"task":"/task/evolutionary-algorithms","name":"Evolutionary Algorithms","papers":2},{"task":"/task/image-classification","name":"Image Classification","papers":2},{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":2},{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":2},{"task":"/task/time-series-1","name":"Time Series","papers":2},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":2},{"task":"/task/benchmarking","name":"Benchmarking","papers":1},{"task":"/task/click-through-rate-prediction","name":"Click-Through Rate Prediction","papers":1},{"task":"/task/diagnostic","name":"Diagnostic","papers":1},{"task":"/task/diversity","name":"Diversity","papers":1},{"task":"/task/hardware-aware-neural-architecture-search","name":"Hardware Aware Neural Architecture Search","papers":1},{"task":"/task/meta-learning","name":"Meta-Learning","papers":1},{"task":"/task/model-compression","name":"Model Compression","papers":1},{"task":"/task/object-detection","name":"Object Detection","papers":1},{"task":"/task/prediction","name":"Prediction","papers":1},{"task":"/task/quantization","name":"Quantization","papers":1},{"task":"/task/scheduling","name":"Scheduling","papers":1},{"task":"/task/sensitivity","name":"Sensitivity","papers":1}],"tasks_shown":20,"n_tasks":28,"usage_by_year":[{"year":"2018","papers":1},{"year":"2019","papers":1},{"year":"2020","papers":2},{"year":"2021","papers":4},{"year":"2022","papers":4},{"year":"2023","papers":3},{"year":"2024","papers":4},{"year":"2025","papers":1}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/dnas"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}