{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/local-search-is-a-remarkably-strong-baseline","title":"Local Search is a Remarkably Strong Baseline for Neural Architecture Search","arxiv_id":"2004.08996","date":"2020-04-20","proceeding":null,"authors":["T. Den Ottelander","A. Dushatskiy","M. Virgolin","P. A. N. Bosman"],"abstract":"Neural Architecture Search (NAS), i.e., the automation of neural network design, has gained much popularity in recent years with increasingly complex search algorithms being proposed. Yet, solid comparisons with simple baselines are often missing. At the same time, recent retrospective studies have found many new algorithms to be no better than random search (RS). In this work we consider, for the first time, a simple Local Search (LS) algorithm for NAS. We particularly consider a multi-objective NAS formulation, with network accuracy and network complexity as two objectives, as understanding the trade-off between these two objectives is arguably the most interesting aspect of NAS. The proposed LS algorithm is compared with RS and two evolutionary algorithms (EAs), as these are often heralded as being ideal for multi-objective optimization. To promote reproducibility, we create and release two benchmark datasets, named MacroNAS-C10 and MacroNAS-C100, containing 200K saved network evaluations for two established image classification tasks, CIFAR-10 and CIFAR-100. Our benchmarks are designed to be complementary to existing benchmarks, especially in that they are better suited for multi-objective search. We additionally consider a version of the problem with a much larger architecture space. While we find and show that the considered algorithms explore the search space in fundamentally different ways, we also find that LS substantially outperforms RS and even performs nearly as good as state-of-the-art EAs. We believe that this provides strong evidence that LS is truly a competitive baseline for NAS against which new NAS algorithms should be benchmarked.","url_abs":"https://arxiv.org/abs/2004.08996v3","url_pdf":"https://arxiv.org/pdf/2004.08996v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"local-search-is-a-remarkably-strong-baseline","repo_url":"https://github.com/ArkadiyD/MacroNASBenchmark","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"BSD-3-Clause"}},{"paper_slug":"local-search-is-a-remarkably-strong-baseline","repo_url":"https://github.com/tdenottelander/MacroNAS","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"tasks":[{"task_slug":"evolutionary-algorithms","task_name":"Evolutionary Algorithms"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"architecture-search","task_name":"Neural Architecture Search"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":"random-search","method_name":"Random Search"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2004.08996","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2004.08996"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ArkadiyD/MacroNASBenchmark","reach":{"status":"ok","spdx":"BSD-3-Clause"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tdenottelander/MacroNAS","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"summary":{"unverified":8},"by_repo_kind":{"official":{"samples":8,"ran":0,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"61203f52b3a1207c","entry":"check_model_exist","repo":"ArkadiyD/MacroNASBenchmark","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/ArkadiyD/MacroNASBenchmark/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"61203f52b3a1207c"}},{"code_sha256_prefix":"ae14944391146c6c","entry":"check_model_exist","repo":"tdenottelander/MacroNAS","repo_kind":"official","path":"NAS_largescale/utils.py","file_url":"https://github.com/tdenottelander/MacroNAS/blob/HEAD/NAS_largescale/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"ae14944391146c6c"}},{"code_sha256_prefix":"062252b5adfda5f3","entry":"get_device","repo":"ArkadiyD/MacroNASBenchmark","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/ArkadiyD/MacroNASBenchmark/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"062252b5adfda5f3"}},{"code_sha256_prefix":"5925caa04b5d66c9","entry":"get_number_of_classes","repo":"tdenottelander/MacroNAS","repo_kind":"official","path":"NAS_largescale/cifar.py","file_url":"https://github.com/tdenottelander/MacroNAS/blob/HEAD/NAS_largescale/cifar.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"5925caa04b5d66c9"}},{"code_sha256_prefix":"7cb8942d2295e814","entry":"get_number_of_input_channels","repo":"tdenottelander/MacroNAS","repo_kind":"official","path":"NAS_largescale/cifar.py","file_url":"https://github.com/tdenottelander/MacroNAS/blob/HEAD/NAS_largescale/cifar.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"7cb8942d2295e814"}},{"code_sha256_prefix":"ac84cd47cff2eb86","entry":"get_size","repo":"tdenottelander/MacroNAS","repo_kind":"official","path":"NAS_largescale/cifar.py","file_url":"https://github.com/tdenottelander/MacroNAS/blob/HEAD/NAS_largescale/cifar.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"ac84cd47cff2eb86"}},{"code_sha256_prefix":"e387a0294aa2d68b","entry":"make_layers","repo":"tdenottelander/MacroNAS","repo_kind":"official","path":"NAS_largescale/model.py","file_url":"https://github.com/tdenottelander/MacroNAS/blob/HEAD/NAS_largescale/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"e387a0294aa2d68b"}},{"code_sha256_prefix":"89824deb97869bdc","entry":"model_to_solution","repo":"ArkadiyD/MacroNASBenchmark","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/ArkadiyD/MacroNASBenchmark/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"89824deb97869bdc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}