{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/l-darts-mitigating-performance-collapse-by","title":"$Λ$-DARTS: Mitigating Performance Collapse by Harmonizing Operation Selection among Cells","arxiv_id":"2210.07998","date":"2022-10-14","proceeding":null,"authors":["Sajad Movahedi","Melika Adabinejad","Ayyoob Imani","Arezou Keshavarz","Mostafa Dehghani","Azadeh Shakery","Babak N. Araabi"],"abstract":"Differentiable neural architecture search (DARTS) is a popular method for neural architecture search (NAS), which performs cell-search and utilizes continuous relaxation to improve the search efficiency via gradient-based optimization. The main shortcoming of DARTS is performance collapse, where the discovered architecture suffers from a pattern of declining quality during search. Performance collapse has become an important topic of research, with many methods trying to solve the issue through either regularization or fundamental changes to DARTS. However, the weight-sharing framework used for cell-search in DARTS and the convergence of architecture parameters has not been analyzed yet. In this paper, we provide a thorough and novel theoretical and empirical analysis on DARTS and its point of convergence. We show that DARTS suffers from a specific structural flaw due to its weight-sharing framework that limits the convergence of DARTS to saturation points of the softmax function. This point of convergence gives an unfair advantage to layers closer to the output in choosing the optimal architecture, causing performance collapse. We then propose two new regularization terms that aim to prevent performance collapse by harmonizing operation selection via aligning gradients of layers. Experimental results on six different search spaces and three different datasets show that our method ($\\Lambda$-DARTS) does indeed prevent performance collapse, providing justification for our theoretical analysis and the proposed remedy.","url_abs":"https://arxiv.org/abs/2210.07998v2","url_pdf":"https://arxiv.org/pdf/2210.07998v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"l-darts-mitigating-performance-collapse-by","repo_url":"https://github.com/dr-faustus/lambda-darts","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"architecture-search","task_name":"Neural Architecture Search"}],"methods":[{"method_slug":"darts","method_name":"DARTS"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/neural-architecture-search-on-nas-bench-201-1","task":"Neural Architecture Search","dataset":"NAS-Bench-201, CIFAR-10","model":"Λ-DARTS","rank_in_archive_order":6,"of":37,"metrics":{"Accuracy (Test)":"94.36","Accuracy (Val)":"91.55"},"uses_additional_data":false},{"leaderboard":"/sota/neural-architecture-search-on-nas-bench-201-2","task":"Neural Architecture Search","dataset":"NAS-Bench-201, CIFAR-100","model":"Λ-DARTS","rank_in_archive_order":6,"of":40,"metrics":{"Accuracy (Test)":"73.51","Accuracy (Val)":"73.49"},"uses_additional_data":false},{"leaderboard":"/sota/neural-architecture-search-on-nas-bench-201","task":"Neural Architecture Search","dataset":"NAS-Bench-201, ImageNet-16-120","model":"Λ-DARTS","rank_in_archive_order":13,"of":49,"metrics":{"Accuracy (Test)":"46.34","Accuracy (Val)":"46.37"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2210.07998","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.07998"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/dr-faustus/Lambda-DARTS","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dr-faustus/lambda-darts","reach":{"status":"ok"}}],"summary":{"unverified":1},"by_repo_kind":{"official":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"490b5c41400816e1","entry":"backward_step_unrolled","repo":"dr-faustus/Lambda-DARTS","repo_kind":"official","path":"NAS-Bench201/exps/algos/DARTS-V2.py","file_url":"https://github.com/dr-faustus/Lambda-DARTS/blob/HEAD/NAS-Bench201/exps/algos/DARTS-V2.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"490b5c41400816e1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}