{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-where-to-look-generative-nas-is","title":"Learning Where To Look -- Generative NAS is Surprisingly Efficient","arxiv_id":"2203.08734","date":"2022-03-16","proceeding":null,"authors":["Jovita Lukasik","Steffen Jung","Margret Keuper"],"abstract":"The efficient, automated search for well-performing neural architectures (NAS) has drawn increasing attention in the recent past. Thereby, the predominant research objective is to reduce the necessity of costly evaluations of neural architectures while efficiently exploring large search spaces. To this aim, surrogate models embed architectures in a latent space and predict their performance, while generative models for neural architectures enable optimization-based search within the latent space the generator draws from. Both, surrogate and generative models, have the aim of facilitating query-efficient search in a well-structured latent space. In this paper, we further improve the trade-off between query-efficiency and promising architecture generation by leveraging advantages from both, efficient surrogate models and generative design. To this end, we propose a generative model, paired with a surrogate predictor, that iteratively learns to generate samples from increasingly promising latent subspaces. This approach leads to very effective and efficient architecture search, while keeping the query amount low. In addition, our approach allows in a straightforward manner to jointly optimize for multiple objectives such as accuracy and hardware latency. We show the benefit of this approach not only w.r.t. the optimization of architectures for highest classification accuracy but also in the context of hardware constraints and outperform state-of-the-art methods on several NAS benchmarks for single and multiple objectives. We also achieve state-of-the-art performance on ImageNet. The code is available at http://github.com/jovitalukasik/AG-Net .","url_abs":"https://arxiv.org/abs/2203.08734v2","url_pdf":"https://arxiv.org/pdf/2203.08734v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-where-to-look-generative-nas-is","repo_url":"https://github.com/jovitalukasik/AG-Net","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"architecture-search","task_name":"Neural Architecture Search"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/neural-architecture-search-on-nas-bench-201-1","task":"Neural Architecture Search","dataset":"NAS-Bench-201, CIFAR-10","model":"AG-Net","rank_in_archive_order":3,"of":37,"metrics":{"Accuracy (Test)":"94.37","Accuracy (Val)":"91.61"},"uses_additional_data":false},{"leaderboard":"/sota/neural-architecture-search-on-nas-bench-201-2","task":"Neural Architecture Search","dataset":"NAS-Bench-201, CIFAR-100","model":"AG-Net","rank_in_archive_order":5,"of":40,"metrics":{"Accuracy (Test)":"73.51","Accuracy (Val)":"73.49"},"uses_additional_data":false},{"leaderboard":"/sota/neural-architecture-search-on-nas-bench-201","task":"Neural Architecture Search","dataset":"NAS-Bench-201, ImageNet-16-120","model":"AG-Net","rank_in_archive_order":7,"of":49,"metrics":{"Accuracy (Test)":"46.42","Accuracy (Val)":"46.73"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2203.08734","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.08734"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/jovitalukasik/AG-Net","reach":null}],"summary":{"ran":6,"ran_fixture":1,"ran_draft_wrong":1,"unverified":7},"by_repo_kind":{"official":{"samples":15,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"582d38d6fe781c5d","entry":"Embedding_Network","repo":"jovitalukasik/AG-Net","repo_kind":"official","path":"Generator.py","file_url":"https://github.com/jovitalukasik/AG-Net/blob/HEAD/Generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"582d38d6fe781c5d"}},{"code_sha256_prefix":"ce4b703f0ca4ac1d","entry":"GraphAggr","repo":"jovitalukasik/AG-Net","repo_kind":"official","path":"Generator.py","file_url":"https://github.com/jovitalukasik/AG-Net/blob/HEAD/Generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ce4b703f0ca4ac1d"}},{"code_sha256_prefix":"6108502415b404b5","entry":"MLP_predictor","repo":"jovitalukasik/AG-Net","repo_kind":"official","path":"Generator.py","file_url":"https://github.com/jovitalukasik/AG-Net/blob/HEAD/Generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6108502415b404b5"}},{"code_sha256_prefix":"2b9fe11164cf5625","entry":"NodeAdd","repo":"jovitalukasik/AG-Net","repo_kind":"official","path":"Generator.py","file_url":"https://github.com/jovitalukasik/AG-Net/blob/HEAD/Generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2b9fe11164cf5625"}},{"code_sha256_prefix":"dad4798b4ede036f","entry":"NodeInit","repo":"jovitalukasik/AG-Net","repo_kind":"official","path":"Generator.py","file_url":"https://github.com/jovitalukasik/AG-Net/blob/HEAD/Generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dad4798b4ede036f"}},{"code_sha256_prefix":"775bc58aa0b13b2c","entry":"Nodes","repo":"jovitalukasik/AG-Net","repo_kind":"official","path":"Generator.py","file_url":"https://github.com/jovitalukasik/AG-Net/blob/HEAD/Generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"775bc58aa0b13b2c"}},{"code_sha256_prefix":"79e9ac44d05a5e89","entry":"adj_to_scores","repo":"jovitalukasik/AG-Net","repo_kind":"official","path":"Generator.py","file_url":"https://github.com/jovitalukasik/AG-Net/blob/HEAD/Generator.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"79e9ac44d05a5e89"}},{"code_sha256_prefix":"07bfd504987b0c00","entry":"scores_to_adj","repo":"jovitalukasik/AG-Net","repo_kind":"official","path":"Generator.py","file_url":"https://github.com/jovitalukasik/AG-Net/blob/HEAD/Generator.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"07bfd504987b0c00"}},{"code_sha256_prefix":"7a4c97aeb50377dc","entry":"GNNDecoder","repo":"jovitalukasik/AG-Net","repo_kind":"official","path":"Generator.py","file_url":"https://github.com/jovitalukasik/AG-Net/blob/HEAD/Generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7a4c97aeb50377dc"}},{"code_sha256_prefix":"2d556bf1eff82766","entry":"GNNLayer_backward","repo":"jovitalukasik/AG-Net","repo_kind":"official","path":"Generator.py","file_url":"https://github.com/jovitalukasik/AG-Net/blob/HEAD/Generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2d556bf1eff82766"}},{"code_sha256_prefix":"010094ca34994468","entry":"GNNLayer_forward","repo":"jovitalukasik/AG-Net","repo_kind":"official","path":"Generator.py","file_url":"https://github.com/jovitalukasik/AG-Net/blob/HEAD/Generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"010094ca34994468"}},{"code_sha256_prefix":"011cde17ef45ee9a","entry":"Generator","repo":"jovitalukasik/AG-Net","repo_kind":"official","path":"Generator.py","file_url":"https://github.com/jovitalukasik/AG-Net/blob/HEAD/Generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"011cde17ef45ee9a"}},{"code_sha256_prefix":"5d7d83be99daf869","entry":"Generator_Decoder","repo":"jovitalukasik/AG-Net","repo_kind":"official","path":"Generator.py","file_url":"https://github.com/jovitalukasik/AG-Net/blob/HEAD/Generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5d7d83be99daf869"}},{"code_sha256_prefix":"57beeabace709277","entry":"GraphEmbed","repo":"jovitalukasik/AG-Net","repo_kind":"official","path":"Generator.py","file_url":"https://github.com/jovitalukasik/AG-Net/blob/HEAD/Generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"57beeabace709277"}},{"code_sha256_prefix":"bfe2ce35559e1337","entry":"NodeEmbUpd","repo":"jovitalukasik/AG-Net","repo_kind":"official","path":"Generator.py","file_url":"https://github.com/jovitalukasik/AG-Net/blob/HEAD/Generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bfe2ce35559e1337"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}