{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/local-search-is-state-of-the-art-for-nas","title":"Exploring the Loss Landscape in Neural Architecture Search","arxiv_id":"2005.02960","date":"2020-05-06","proceeding":null,"authors":["Colin White","Sam Nolen","Yash Savani"],"abstract":"Neural architecture search (NAS) has seen a steep rise in interest over the last few years. Many algorithms for NAS consist of searching through a space of architectures by iteratively choosing an architecture, evaluating its performance by training it, and using all prior evaluations to come up with the next choice. The evaluation step is noisy - the final accuracy varies based on the random initialization of the weights. Prior work has focused on devising new search algorithms to handle this noise, rather than quantifying or understanding the level of noise in architecture evaluations. In this work, we show that (1) the simplest hill-climbing algorithm is a powerful baseline for NAS, and (2), when the noise in popular NAS benchmark datasets is reduced to a minimum, hill-climbing to outperforms many popular state-of-the-art algorithms. We further back up this observation by showing that the number of local minima is substantially reduced as the noise decreases, and by giving a theoretical characterization of the performance of local search in NAS. Based on our findings, for NAS research we suggest (1) using local search as a baseline, and (2) denoising the training pipeline when possible.","url_abs":"https://arxiv.org/abs/2005.02960v3","url_pdf":"https://arxiv.org/pdf/2005.02960v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"local-search-is-state-of-the-art-for-nas","repo_url":"https://github.com/naszilla/naszilla","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"local-search-is-state-of-the-art-for-nas","repo_url":"https://github.com/realityengines/local_search","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}}],"tasks":[{"task_slug":"combinatorial-optimization","task_name":"Combinatorial Optimization"},{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"hyperparameter-optimization","task_name":"Hyperparameter Optimization"},{"task_slug":"architecture-search","task_name":"Neural Architecture Search"},{"task_slug":"traveling-salesman-problem","task_name":"Traveling Salesman Problem"}],"methods":[{"method_slug":"darts","method_name":"DARTS"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/neural-architecture-search-on-nas-bench-201","task":"Neural Architecture Search","dataset":"NAS-Bench-201, ImageNet-16-120","model":"Local search","rank_in_archive_order":8,"of":49,"metrics":{"Accuracy (Test)":"46.38","Search time (s)":"151200"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2005.02960","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2005.02960"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/naszilla/naszilla","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/realityengines/local_search","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":2},"by_repo_kind":{},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"872244025f86be63","entry":"algo_params","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"872244025f86be63"}},{"code_sha256_prefix":"450f026eb158bc17","entry":"meta_neuralnet_params","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"450f026eb158bc17"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}