{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/asap-architecture-search-anneal-and-prune","title":"ASAP: Architecture Search, Anneal and Prune","arxiv_id":"1904.04123","date":"2019-04-08","proceeding":null,"authors":["Asaf Noy","Niv Nayman","Tal Ridnik","Nadav Zamir","Sivan Doveh","Itamar Friedman","Raja Giryes","Lihi Zelnik-Manor"],"abstract":"Automatic methods for Neural Architecture Search (NAS) have been shown to produce state-of-the-art network models. Yet, their main drawback is the computational complexity of the search process. As some primal methods optimized over a discrete search space, thousands of days of GPU were required for convergence. A recent approach is based on constructing a differentiable search space that enables gradient-based optimization, which reduces the search time to a few days. While successful, it still includes some noncontinuous steps, e.g., the pruning of many weak connections at once. In this paper, we propose a differentiable search space that allows the annealing of architecture weights, while gradually pruning inferior operations. In this way, the search converges to a single output network in a continuous manner. Experiments on several vision datasets demonstrate the effectiveness of our method with respect to the search cost and accuracy of the achieved model. Specifically, with $0.2$ GPU search days we achieve an error rate of $1.68\\%$ on CIFAR-10.","url_abs":"https://arxiv.org/abs/1904.04123v2","url_pdf":"https://arxiv.org/pdf/1904.04123v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"asap-architecture-search-anneal-and-prune","repo_url":"https://github.com/cogsys-tuebingen/uninas","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":null,"task_name":"GPU"},{"task_slug":"architecture-search","task_name":"Neural Architecture Search"}],"methods":[{"method_slug":"cosine-annealing","method_name":"Cosine Annealing"},{"method_slug":"cosine-power-annealing","method_name":"Cosine Power Annealing"},{"method_slug":"exponential-decay","method_name":"Exponential Decay"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"pruning","method_name":"Pruning"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1904.04123","atlas_url":"https://app.syntology.ai/?focus=1904.04123","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}