{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/theory-inspired-path-regularized-differential","title":"Theory-Inspired Path-Regularized Differential Network Architecture Search","arxiv_id":"2006.16537","date":"2020-06-30","proceeding":"NeurIPS 2020 12","authors":["Pan Zhou","Caiming Xiong","Richard Socher","Steven C. H. Hoi"],"abstract":"Despite its high search efficiency, differential architecture search (DARTS) often selects network architectures with dominated skip connections which lead to performance degradation. However, theoretical understandings on this issue remain absent, hindering the development of more advanced methods in a principled way. In this work, we solve this problem by theoretically analyzing the effects of various types of operations, e.g. convolution, skip connection and zero operation, to the network optimization. We prove that the architectures with more skip connections can converge faster than the other candidates, and thus are selected by DARTS. This result, for the first time, theoretically and explicitly reveals the impact of skip connections to fast network optimization and its competitive advantage over other types of operations in DARTS. Then we propose a theory-inspired path-regularized DARTS that consists of two key modules: (i) a differential group-structured sparse binary gate introduced for each operation to avoid unfair competition among operations, and (ii) a path-depth-wise regularization used to incite search exploration for deep architectures that often converge slower than shallow ones as shown in our theory and are not well explored during the search. Experimental results on image classification tasks validate its advantages.","url_abs":"https://arxiv.org/abs/2006.16537v2","url_pdf":"https://arxiv.org/pdf/2006.16537v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"theory-inspired-path-regularized-differential","repo_url":"https://github.com/salesforce/PR-DARTS","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":"darts","method_name":"DARTS"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2006.16537","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2006.16537"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/salesforce/PR-DARTS","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"unverified":5},"by_repo_kind":{"listed":{"samples":6,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e7ef53e9c5847e3e","entry":"adjust_lr","repo":"salesforce/PR-DARTS","repo_kind":"listed","path":"PRDARTS_eval/train_imagenet.py","file_url":"https://github.com/salesforce/PR-DARTS/blob/HEAD/PRDARTS_eval/train_imagenet.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e7ef53e9c5847e3e"}},{"code_sha256_prefix":"469bf48752905889","entry":"accuracy","repo":"salesforce/PR-DARTS","repo_kind":"listed","path":"PRDARTS_eval/utils.py","file_url":"https://github.com/salesforce/PR-DARTS/blob/HEAD/PRDARTS_eval/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"469bf48752905889"}},{"code_sha256_prefix":"11f5fc83cf977b27","entry":"change_key","repo":"salesforce/PR-DARTS","repo_kind":"listed","path":"PRDARTS_search/get_cell_based_tiny_net.py","file_url":"https://github.com/salesforce/PR-DARTS/blob/HEAD/PRDARTS_search/get_cell_based_tiny_net.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"11f5fc83cf977b27"}},{"code_sha256_prefix":"3e816289579cc9e9","entry":"count_parameters_in_MB","repo":"salesforce/PR-DARTS","repo_kind":"listed","path":"PRDARTS_eval/utils.py","file_url":"https://github.com/salesforce/PR-DARTS/blob/HEAD/PRDARTS_eval/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3e816289579cc9e9"}},{"code_sha256_prefix":"13c2719404e25b25","entry":"drop_path","repo":"salesforce/PR-DARTS","repo_kind":"listed","path":"PRDARTS_eval/utils.py","file_url":"https://github.com/salesforce/PR-DARTS/blob/HEAD/PRDARTS_eval/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"13c2719404e25b25"}},{"code_sha256_prefix":"d5744b71dab29e1e","entry":"parse_channel_info","repo":"salesforce/PR-DARTS","repo_kind":"listed","path":"PRDARTS_search/get_cell_based_tiny_net.py","file_url":"https://github.com/salesforce/PR-DARTS/blob/HEAD/PRDARTS_search/get_cell_based_tiny_net.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d5744b71dab29e1e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}