{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/robustifying-and-boosting-training-free","title":"Robustifying and Boosting Training-Free Neural Architecture Search","arxiv_id":"2403.07591","date":"2024-03-12","proceeding":null,"authors":["Zhenfeng He","Yao Shu","Zhongxiang Dai","Bryan Kian Hsiang Low"],"abstract":"Neural architecture search (NAS) has become a key component of AutoML and a standard tool to automate the design of deep neural networks. Recently, training-free NAS as an emerging paradigm has successfully reduced the search costs of standard training-based NAS by estimating the true architecture performance with only training-free metrics. Nevertheless, the estimation ability of these metrics typically varies across different tasks, making it challenging to achieve robust and consistently good search performance on diverse tasks with only a single training-free metric. Meanwhile, the estimation gap between training-free metrics and the true architecture performances limits training-free NAS to achieve superior performance. To address these challenges, we propose the robustifying and boosting training-free NAS (RoBoT) algorithm which (a) employs the optimized combination of existing training-free metrics explored from Bayesian optimization to develop a robust and consistently better-performing metric on diverse tasks, and (b) applies greedy search, i.e., the exploitation, on the newly developed metric to bridge the aforementioned gap and consequently to boost the search performance of standard training-free NAS further. Remarkably, the expected performance of our RoBoT can be theoretically guaranteed, which improves over the existing training-free NAS under mild conditions with additional interesting insights. Our extensive experiments on various NAS benchmark tasks yield substantial empirical evidence to support our theoretical results.","url_abs":"https://arxiv.org/abs/2403.07591v1","url_pdf":"https://arxiv.org/pdf/2403.07591v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"robustifying-and-boosting-training-free","repo_url":"https://github.com/hzf1174/robot","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"automl","task_name":"AutoML"},{"task_slug":"bayesian-optimization","task_name":"Bayesian Optimization"},{"task_slug":"architecture-search","task_name":"Neural Architecture Search"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.07591","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.07591"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/hzf1174/RoBoT","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hzf1174/robot","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":2,"ran_violates":2,"ran_fixture":1,"ran":2,"ran_honours":2,"unverified":6},"by_repo_kind":{"official":{"samples":15,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ad0636a128295d5a","entry":"calculate_md5","repo":"hzf1174/RoBoT","repo_kind":"official","path":"foresight/imagenet16.py","file_url":"https://github.com/hzf1174/RoBoT/blob/HEAD/foresight/imagenet16.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ad0636a128295d5a"}},{"code_sha256_prefix":"380a00b777aad655","entry":"check_md5","repo":"hzf1174/RoBoT","repo_kind":"official","path":"foresight/imagenet16.py","file_url":"https://github.com/hzf1174/RoBoT/blob/HEAD/foresight/imagenet16.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":2,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"380a00b777aad655"}},{"code_sha256_prefix":"eab4f74c38cc3c8f","entry":"accuracy","repo":"hzf1174/RoBoT","repo_kind":"official","path":"darts_space/utils.py","file_url":"https://github.com/hzf1174/RoBoT/blob/HEAD/darts_space/utils.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"eab4f74c38cc3c8f"}},{"code_sha256_prefix":"e127535d5e962cac","entry":"adjust_lr","repo":"hzf1174/RoBoT","repo_kind":"official","path":"darts_space/train_imagenet.py","file_url":"https://github.com/hzf1174/RoBoT/blob/HEAD/darts_space/train_imagenet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e127535d5e962cac"}},{"code_sha256_prefix":"e947a6effcaaf32b","entry":"check_integrity","repo":"hzf1174/RoBoT","repo_kind":"official","path":"foresight/imagenet16.py","file_url":"https://github.com/hzf1174/RoBoT/blob/HEAD/foresight/imagenet16.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e947a6effcaaf32b"}},{"code_sha256_prefix":"c1f272141fc24942","entry":"copynet","repo":"hzf1174/robot","repo_kind":"official","path":"foresight/pruners/predictive.py","file_url":"https://github.com/hzf1174/robot/blob/HEAD/foresight/pruners/predictive.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c1f272141fc24942"}},{"code_sha256_prefix":"e615a9d09865bd9f","entry":"count_parameters_in_MB","repo":"hzf1174/RoBoT","repo_kind":"official","path":"darts_space/utils.py","file_url":"https://github.com/hzf1174/RoBoT/blob/HEAD/darts_space/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e615a9d09865bd9f"}},{"code_sha256_prefix":"63f0914560e34c40","entry":"no_op","repo":"hzf1174/robot","repo_kind":"official","path":"foresight/pruners/predictive.py","file_url":"https://github.com/hzf1174/robot/blob/HEAD/foresight/pruners/predictive.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"63f0914560e34c40"}},{"code_sha256_prefix":"7f332db65cde5273","entry":"sum_arr","repo":"hzf1174/robot","repo_kind":"official","path":"darts_space/search.py","file_url":"https://github.com/hzf1174/robot/blob/HEAD/darts_space/search.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7f332db65cde5273"}},{"code_sha256_prefix":"2c6bc306e5f9c38b","entry":"drop_path","repo":"hzf1174/RoBoT","repo_kind":"official","path":"darts_space/utils.py","file_url":"https://github.com/hzf1174/RoBoT/blob/HEAD/darts_space/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2c6bc306e5f9c38b"}},{"code_sha256_prefix":"bfe0770011ad29c4","entry":"find_measures","repo":"hzf1174/robot","repo_kind":"official","path":"foresight/pruners/predictive.py","file_url":"https://github.com/hzf1174/robot/blob/HEAD/foresight/pruners/predictive.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bfe0770011ad29c4"}},{"code_sha256_prefix":"20066c8b8cca9c9a","entry":"find_measures_arrays","repo":"hzf1174/robot","repo_kind":"official","path":"foresight/pruners/predictive.py","file_url":"https://github.com/hzf1174/robot/blob/HEAD/foresight/pruners/predictive.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"20066c8b8cca9c9a"}},{"code_sha256_prefix":"c8dca8dc011090bb","entry":"get_cifar_dataloaders","repo":"hzf1174/RoBoT","repo_kind":"official","path":"foresight/dataset.py","file_url":"https://github.com/hzf1174/RoBoT/blob/HEAD/foresight/dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c8dca8dc011090bb"}},{"code_sha256_prefix":"7491772a906c4046","entry":"get_mnist_dataloaders","repo":"hzf1174/RoBoT","repo_kind":"official","path":"foresight/dataset.py","file_url":"https://github.com/hzf1174/RoBoT/blob/HEAD/foresight/dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7491772a906c4046"}},{"code_sha256_prefix":"a9a567af7564f3f0","entry":"search","repo":"hzf1174/robot","repo_kind":"official","path":"search_nb201.py","file_url":"https://github.com/hzf1174/robot/blob/HEAD/search_nb201.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a9a567af7564f3f0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}