{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rapid-neural-architecture-search-by-learning-1","title":"Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets","arxiv_id":"2107.00860","date":"2021-07-02","proceeding":"ICLR 2021 1","authors":["Hayeon Lee","Eunyoung Hyung","Sung Ju Hwang"],"abstract":"Despite the success of recent Neural Architecture Search (NAS) methods on various tasks which have shown to output networks that largely outperform human-designed networks, conventional NAS methods have mostly tackled the optimization of searching for the network architecture for a single task (dataset), which does not generalize well across multiple tasks (datasets). Moreover, since such task-specific methods search for a neural architecture from scratch for every given task, they incur a large computational cost, which is problematic when the time and monetary budget are limited. In this paper, we propose an efficient NAS framework that is trained once on a database consisting of datasets and pretrained networks and can rapidly search for a neural architecture for a novel dataset. The proposed MetaD2A (Meta Dataset-to-Architecture) model can stochastically generate graphs (architectures) from a given set (dataset) via a cross-modal latent space learned with amortized meta-learning. Moreover, we also propose a meta-performance predictor to estimate and select the best architecture without direct training on target datasets. The experimental results demonstrate that our model meta-learned on subsets of ImageNet-1K and architectures from NAS-Bench 201 search space successfully generalizes to multiple unseen datasets including CIFAR-10 and CIFAR-100, with an average search time of 33 GPU seconds. Even under MobileNetV3 search space, MetaD2A is 5.5K times faster than NSGANetV2, a transferable NAS method, with comparable performance. We believe that the MetaD2A proposes a new research direction for rapid NAS as well as ways to utilize the knowledge from rich databases of datasets and architectures accumulated over the past years. Code is available at https://github.com/HayeonLee/MetaD2A.","url_abs":"https://arxiv.org/abs/2107.00860v1","url_pdf":"https://arxiv.org/pdf/2107.00860v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rapid-neural-architecture-search-by-learning-1","repo_url":"https://github.com/HayeonLee/MetaD2A","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":null,"task_name":"GPU"},{"task_slug":"meta-learning","task_name":"Meta-Learning"},{"task_slug":"architecture-search","task_name":"Neural Architecture Search"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"depthwise-convolution","method_name":"Depthwise Convolution"},{"method_slug":"depthwise-separable-convolution","method_name":"Depthwise Separable Convolution"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"hard-swish","method_name":"Hard Swish"},{"method_slug":"inverted-residual-block","method_name":"Inverted Residual Block"},{"method_slug":"pointwise-convolution","method_name":"Pointwise Convolution"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"relu6","method_name":"ReLU6"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"squeeze-and-excitation-block","method_name":"Squeeze-and-Excitation Block"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2107.00860","atlas_url":"https://app.syntology.ai/?focus=2107.00860","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2107.00860"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/xiangning-chen/DrNAS","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/HayeonLee/MetaD2A","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/yuhuixu1993/PC-DARTS","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/D-X-Y/AutoDL-Projects","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"ran_honours":1,"unverified":17},"by_repo_kind":{"official":{"samples":9,"ran":1,"repositories":1},"found_in_text":{"samples":9,"ran":0,"repositories":3}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"b9da06d4f527dd6c","entry":"channel_shuffle","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"b9da06d4f527dd6c"}},{"code_sha256_prefix":"f55c31d782e48ffb","entry":"collate_fn","repo":"HayeonLee/MetaD2A","repo_kind":"official","path":"MetaD2A_mobilenetV3/loader.py","file_url":"https://github.com/HayeonLee/MetaD2A/blob/HEAD/MetaD2A_mobilenetV3/loader.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f55c31d782e48ffb"}},{"code_sha256_prefix":"461bb7a03f47f0ad","entry":"Architect","repo":"xiangning-chen/DrNAS","repo_kind":"found_in_text","path":"architect.py","file_url":"https://github.com/xiangning-chen/DrNAS/blob/HEAD/architect.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"461bb7a03f47f0ad"}},{"code_sha256_prefix":"18a2cee21566ce44","entry":"Cell","repo":"yuhuixu1993/PC-DARTS","repo_kind":"found_in_text","path":"model_search.py","file_url":"https://github.com/yuhuixu1993/PC-DARTS/blob/HEAD/model_search.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"18a2cee21566ce44"}},{"code_sha256_prefix":"f58cf0ad967f722d","entry":"MixedOp","repo":"yuhuixu1993/PC-DARTS","repo_kind":"found_in_text","path":"model_search.py","file_url":"https://github.com/yuhuixu1993/PC-DARTS/blob/HEAD/model_search.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f58cf0ad967f722d"}},{"code_sha256_prefix":"4b9ef6a87294337c","entry":"Network","repo":"yuhuixu1993/PC-DARTS","repo_kind":"found_in_text","path":"model_search.py","file_url":"https://github.com/yuhuixu1993/PC-DARTS/blob/HEAD/model_search.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4b9ef6a87294337c"}},{"code_sha256_prefix":"490b5c41400816e1","entry":"backward_step_unrolled","repo":"D-X-Y/AutoDL-Projects","repo_kind":"found_in_text","path":"exps/NAS-Bench-201-algos/DARTS-V2.py","file_url":"https://github.com/D-X-Y/AutoDL-Projects/blob/HEAD/exps/NAS-Bench-201-algos/DARTS-V2.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"490b5c41400816e1"}},{"code_sha256_prefix":"7b60cfea1c582309","entry":"decode_ofa_mbv3_to_igraph","repo":"HayeonLee/MetaD2A","repo_kind":"official","path":"MetaD2A_mobilenetV3/utils.py","file_url":"https://github.com/HayeonLee/MetaD2A/blob/HEAD/MetaD2A_mobilenetV3/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7b60cfea1c582309"}},{"code_sha256_prefix":"14ef56df31b716ed","entry":"get_loader","repo":"HayeonLee/MetaD2A","repo_kind":"official","path":"MetaD2A_mobilenetV3/database/metaloader.py","file_url":"https://github.com/HayeonLee/MetaD2A/blob/HEAD/MetaD2A_mobilenetV3/database/metaloader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"14ef56df31b716ed"}},{"code_sha256_prefix":"2d30a7e1cec63917","entry":"get_meta_test_loader","repo":"HayeonLee/MetaD2A","repo_kind":"official","path":"MetaD2A_mobilenetV3/loader.py","file_url":"https://github.com/HayeonLee/MetaD2A/blob/HEAD/MetaD2A_mobilenetV3/loader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2d30a7e1cec63917"}},{"code_sha256_prefix":"75092bae3fb86ad0","entry":"get_meta_train_loader","repo":"HayeonLee/MetaD2A","repo_kind":"official","path":"MetaD2A_mobilenetV3/loader.py","file_url":"https://github.com/HayeonLee/MetaD2A/blob/HEAD/MetaD2A_mobilenetV3/loader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"75092bae3fb86ad0"}},{"code_sha256_prefix":"f2c3cad885f3c5a6","entry":"get_transform","repo":"HayeonLee/MetaD2A","repo_kind":"official","path":"MetaD2A_mobilenetV3/process_dataset.py","file_url":"https://github.com/HayeonLee/MetaD2A/blob/HEAD/MetaD2A_mobilenetV3/process_dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f2c3cad885f3c5a6"}},{"code_sha256_prefix":"0de0dc03a084ba91","entry":"is_valid_ofa_mbv3","repo":"HayeonLee/MetaD2A","repo_kind":"official","path":"MetaD2A_mobilenetV3/utils.py","file_url":"https://github.com/HayeonLee/MetaD2A/blob/HEAD/MetaD2A_mobilenetV3/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0de0dc03a084ba91"}},{"code_sha256_prefix":"dac9f0aab7051ab2","entry":"load_graph_config","repo":"HayeonLee/MetaD2A","repo_kind":"official","path":"MetaD2A_mobilenetV3/utils.py","file_url":"https://github.com/HayeonLee/MetaD2A/blob/HEAD/MetaD2A_mobilenetV3/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dac9f0aab7051ab2"}},{"code_sha256_prefix":"c9db316e18458ccd","entry":"mutate_arch_func","repo":"D-X-Y/AutoDL-Projects","repo_kind":"found_in_text","path":"exps/NAS-Bench-201-algos/R_EA.py","file_url":"https://github.com/D-X-Y/AutoDL-Projects/blob/HEAD/exps/NAS-Bench-201-algos/R_EA.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c9db316e18458ccd"}},{"code_sha256_prefix":"b45190c694bcb277","entry":"random_size_func","repo":"D-X-Y/AutoDL-Projects","repo_kind":"found_in_text","path":"exps/NATS-algos/random_wo_share.py","file_url":"https://github.com/D-X-Y/AutoDL-Projects/blob/HEAD/exps/NATS-algos/random_wo_share.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b45190c694bcb277"}},{"code_sha256_prefix":"0e6f925ca93551fa","entry":"select_action","repo":"D-X-Y/AutoDL-Projects","repo_kind":"found_in_text","path":"exps/NAS-Bench-201-algos/reinforce.py","file_url":"https://github.com/D-X-Y/AutoDL-Projects/blob/HEAD/exps/NAS-Bench-201-algos/reinforce.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0e6f925ca93551fa"}},{"code_sha256_prefix":"b0717d29ccfec107","entry":"str2bool","repo":"HayeonLee/MetaD2A","repo_kind":"official","path":"MetaD2A_mobilenetV3/parser.py","file_url":"https://github.com/HayeonLee/MetaD2A/blob/HEAD/MetaD2A_mobilenetV3/parser.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b0717d29ccfec107"}},{"code_sha256_prefix":"1ab9fd355247f264","entry":"train_and_eval","repo":"D-X-Y/AutoDL-Projects","repo_kind":"found_in_text","path":"exps/NAS-Bench-201-algos/R_EA.py","file_url":"https://github.com/D-X-Y/AutoDL-Projects/blob/HEAD/exps/NAS-Bench-201-algos/R_EA.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1ab9fd355247f264"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}