{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sharpdarts-faster-and-more-accurate","title":"sharpDARTS: Faster and More Accurate Differentiable Architecture Search","arxiv_id":"1903.09900","date":"2019-03-23","proceeding":null,"authors":["Andrew Hundt","Varun Jain","Gregory D. Hager"],"abstract":"Neural Architecture Search (NAS) has been a source of dramatic improvements\nin neural network design, with recent results meeting or exceeding the\nperformance of hand-tuned architectures. However, our understanding of how to\nrepresent the search space for neural net architectures and how to search that\nspace efficiently are both still in their infancy.\n  We have performed an in-depth analysis to identify limitations in a widely\nused search space and a recent architecture search method, Differentiable\nArchitecture Search (DARTS). These findings led us to introduce novel network\nblocks with a more general, balanced, and consistent design; a better-optimized\nCosine Power Annealing learning rate schedule; and other improvements. Our\nresulting sharpDARTS search is 50% faster with a 20-30% relative improvement in\nfinal model error on CIFAR-10 when compared to DARTS. Our best single model run\nhas 1.93% (1.98+/-0.07) validation error on CIFAR-10 and 5.5% error (5.8+/-0.3)\non the recently released CIFAR-10.1 test set. To our knowledge, both are state\nof the art for models of similar size. This model also generalizes\ncompetitively to ImageNet at 25.1% top-1 (7.8% top-5) error.\n  We found improvements for existing search spaces but does DARTS generalize to\nnew domains? We propose Differentiable Hyperparameter Grid Search and the\nHyperCuboid search space, which are representations designed to leverage DARTS\nfor more general parameter optimization. Here we find that DARTS fails to\ngeneralize when compared against a human's one shot choice of models. We look\nback to the DARTS and sharpDARTS search spaces to understand why, and an\nablation study reveals an unusual generalization gap. We finally propose Max-W\nregularization to solve this problem, which proves significantly better than\nthe handmade design. Code will be made available.","url_abs":"http://arxiv.org/abs/1903.09900v1","url_pdf":"http://arxiv.org/pdf/1903.09900v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sharpdarts-faster-and-more-accurate","repo_url":"https://github.com/ahundt/sharpDARTS","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"sharpdarts-faster-and-more-accurate","repo_url":"https://github.com/yangyucheng000/Paper-2/tree/main/SharpDRO-ms-main","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"hyperparameter-optimization","task_name":"Hyperparameter Optimization"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"architecture-search","task_name":"Neural Architecture Search"}],"methods":[{"method_slug":"cosine-power-annealing","method_name":"Cosine Power Annealing"},{"method_slug":"darts","method_name":"DARTS"},{"method_slug":"darts-max-w","method_name":"DARTS Max-W"},{"method_slug":"differentiable-hyperparameter-search","method_name":"Differentiable Hyperparameter Search"},{"method_slug":"exponential-decay","method_name":"Exponential Decay"}],"datasets_introduced":[],"methods_introduced":[{"slug":"differentiable-hyperparameter-search","name":"Differentiable Hyperparameter Search","full_name":"Differentiable Hyperparameter Search"}],"results":[{"leaderboard":"/sota/neural-architecture-search-on-cifar-10","task":"Neural Architecture Search","dataset":"CIFAR-10","model":"SharpSepConvDARTS","rank_in_archive_order":6,"of":41,"metrics":{"FLOPS":"579M","Parameters":"3.6M","Search Time (GPU days)":"0.8","Top-1 Error Rate":"1.98%"},"uses_additional_data":false},{"leaderboard":"/sota/neural-architecture-search-on-cifar-10","task":"Neural Architecture Search","dataset":"CIFAR-10","model":"sharpDARTS","rank_in_archive_order":10,"of":41,"metrics":{"FLOPS":"357M","Parameters":"1.98M","Search Time (GPU days)":"1.8","Top-1 Error Rate":"2.29%"},"uses_additional_data":false},{"leaderboard":"/sota/architecture-search-on-cifar-10-image","task":"Neural Architecture Search","dataset":"CIFAR-10 Image Classification","model":"SharpSepConvDARTS","rank_in_archive_order":4,"of":19,"metrics":{"FLOPS":"579M","Params":"3.6M","Percentage error":"1.98"},"uses_additional_data":false},{"leaderboard":"/sota/neural-architecture-search-on-imagenet","task":"Neural Architecture Search","dataset":"ImageNet","model":"sharpDARTS","rank_in_archive_order":95,"of":135,"metrics":{"Accuracy":"76.0","MACs":"950M","Params":"8.3M","Top-1 Error Rate":"24.0"},"uses_additional_data":false},{"leaderboard":"/sota/neural-architecture-search-on-imagenet","task":"Neural Architecture Search","dataset":"ImageNet","model":"SharpSepConvDARTS","rank_in_archive_order":116,"of":135,"metrics":{"Accuracy":"74.1","MACs":"573M","Params":"4.9M","Top-1 Error Rate":"25.1"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1903.09900","atlas_url":"https://app.syntology.ai/?focus=1903.09900","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1903.09900"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yangyucheng000/Paper-2/tree/main/SharpDRO-ms-main","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ahundt/sharpDARTS","reach":null}],"summary":{"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"bfa400764d6e3fa8","entry":"gen_greedy_path","repo":"ahundt/sharpDARTS","repo_kind":"official","path":"cnn/train_search.py","file_url":"https://github.com/ahundt/sharpDARTS/blob/HEAD/cnn/train_search.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bfa400764d6e3fa8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}