{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/atomnas-fine-grained-end-to-end-neural-1","title":"AtomNAS: Fine-Grained End-to-End Neural Architecture Search","arxiv_id":"1912.09640","date":"2019-12-20","proceeding":"ICLR 2020 1","authors":["Jieru Mei","Yingwei Li","Xiaochen Lian","Xiaojie Jin","Linjie Yang","Alan Yuille","Jianchao Yang"],"abstract":"Search space design is very critical to neural architecture search (NAS) algorithms. We propose a fine-grained search space comprised of atomic blocks, a minimal search unit that is much smaller than the ones used in recent NAS algorithms. This search space allows a mix of operations by composing different types of atomic blocks, while the search space in previous methods only allows homogeneous operations. Based on this search space, we propose a resource-aware architecture search framework which automatically assigns the computational resources (e.g., output channel numbers) for each operation by jointly considering the performance and the computational cost. In addition, to accelerate the search process, we propose a dynamic network shrinkage technique which prunes the atomic blocks with negligible influence on outputs on the fly. Instead of a search-and-retrain two-stage paradigm, our method simultaneously searches and trains the target architecture. Our method achieves state-of-the-art performance under several FLOPs configurations on ImageNet with a small searching cost. We open our entire codebase at: https://github.com/meijieru/AtomNAS.","url_abs":"https://arxiv.org/abs/1912.09640v2","url_pdf":"https://arxiv.org/pdf/1912.09640v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"atomnas-fine-grained-end-to-end-neural-1","repo_url":"https://github.com/meijieru/AtomNAS","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"architecture-search","task_name":"Neural Architecture Search"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/neural-architecture-search-on-imagenet","task":"Neural Architecture Search","dataset":"ImageNet","model":"AtomNAS-C+†","rank_in_archive_order":61,"of":135,"metrics":{"Accuracy":"77.6","MACs":"363M","Params":"5.9M","Top-1 Error Rate":"22.4"},"uses_additional_data":false},{"leaderboard":"/sota/neural-architecture-search-on-imagenet","task":"Neural Architecture Search","dataset":"ImageNet","model":"AtomNAS-B+†","rank_in_archive_order":68,"of":135,"metrics":{"Accuracy":"77.2","MACs":"329M","Params":"5.5M","Top-1 Error Rate":"22.8"},"uses_additional_data":false},{"leaderboard":"/sota/neural-architecture-search-on-imagenet","task":"Neural Architecture Search","dataset":"ImageNet","model":"AtomNAS-A+†","rank_in_archive_order":87,"of":135,"metrics":{"Accuracy":"76.3","MACs":"260M","Params":"4.7M","Top-1 Error Rate":"23.7"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1912.09640","atlas_url":"https://app.syntology.ai/?focus=1912.09640","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}