{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-transferable-architectures-for","title":"Learning Transferable Architectures for Scalable Image Recognition","arxiv_id":"1707.07012","date":"2017-07-21","proceeding":"CVPR 2018 6","authors":["Barret Zoph","Vijay Vasudevan","Jonathon Shlens","Quoc V. Le"],"abstract":"Developing neural network image classification models often requires\nsignificant architecture engineering. In this paper, we study a method to learn\nthe model architectures directly on the dataset of interest. As this approach\nis expensive when the dataset is large, we propose to search for an\narchitectural building block on a small dataset and then transfer the block to\na larger dataset. The key contribution of this work is the design of a new\nsearch space (the \"NASNet search space\") which enables transferability. In our\nexperiments, we search for the best convolutional layer (or \"cell\") on the\nCIFAR-10 dataset and then apply this cell to the ImageNet dataset by stacking\ntogether more copies of this cell, each with their own parameters to design a\nconvolutional architecture, named \"NASNet architecture\". We also introduce a\nnew regularization technique called ScheduledDropPath that significantly\nimproves generalization in the NASNet models. On CIFAR-10 itself, NASNet\nachieves 2.4% error rate, which is state-of-the-art. On ImageNet, NASNet\nachieves, among the published works, state-of-the-art accuracy of 82.7% top-1\nand 96.2% top-5 on ImageNet. Our model is 1.2% better in top-1 accuracy than\nthe best human-invented architectures while having 9 billion fewer FLOPS - a\nreduction of 28% in computational demand from the previous state-of-the-art\nmodel. When evaluated at different levels of computational cost, accuracies of\nNASNets exceed those of the state-of-the-art human-designed models. For\ninstance, a small version of NASNet also achieves 74% top-1 accuracy, which is\n3.1% better than equivalently-sized, state-of-the-art models for mobile\nplatforms. Finally, the learned features by NASNet used with the Faster-RCNN\nframework surpass state-of-the-art by 4.0% achieving 43.1% mAP on the COCO\ndataset.","url_abs":"http://arxiv.org/abs/1707.07012v4","url_pdf":"http://arxiv.org/pdf/1707.07012v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-transferable-architectures-for","repo_url":"https://github.com/Tingelam/tianchi_gd_defect_pytorch_round1","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"learning-transferable-architectures-for","repo_url":"https://github.com/asad-62/IVP-DNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"learning-transferable-architectures-for","repo_url":"https://github.com/johannesu/NASNet-keras","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"learning-transferable-architectures-for","repo_url":"https://github.com/mrzzy/np-dl-assign-1","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"learning-transferable-architectures-for","repo_url":"https://github.com/osmr/imgclsmob","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"mxnet","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"learning-transferable-architectures-for","repo_url":"https://github.com/tally0818/NASNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"learning-transferable-architectures-for","repo_url":"https://github.com/tensorflow/models","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"learning-transferable-architectures-for","repo_url":"https://github.com/tensorflow/models/tree/master/research/slim","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"learning-transferable-architectures-for","repo_url":"https://github.com/titu1994/neural-architecture-search","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"learning-transferable-architectures-for","repo_url":"https://github.com/xufanxiong/classification-of-caltech-256","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"learning-transferable-architectures-for","repo_url":"https://github.com/2024-MindSpore-1/Code5/tree/main/ArbitraryStyleTransfer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"learning-transferable-architectures-for","repo_url":"https://github.com/2024-MindSpore-1/Code6/tree/main/ArbitraryStyleTransfer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"learning-transferable-architectures-for","repo_url":"https://github.com/DataCanvasIO/Hypernets","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"learning-transferable-architectures-for","repo_url":"https://github.com/MarSaKi/nasnet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"learning-transferable-architectures-for","repo_url":"https://github.com/MindSpore-paper-code-3/code1/tree/main/ArbitraryStyleTransfer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"learning-transferable-architectures-for","repo_url":"https://github.com/mindspore-ai/models/tree/master/official/cv/nasnet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"learning-transferable-architectures-for","repo_url":"https://github.com/mindspore-ecosystem/mindcv/blob/main/mindcv/models/nasnet.py","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"architecture-search","task_name":"Neural Architecture Search"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"entropy-regularization","method_name":"Entropy Regularization"},{"method_slug":"exponential-decay","method_name":"Exponential Decay"},{"method_slug":"instance-normalization","method_name":"Instance Normalization"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"neural-architecture-search","method_name":"Neural Architecture Search"},{"method_slug":"ppo","method_name":"PPO"},{"method_slug":"rmsprop","method_name":"RMSProp"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"sgd-with-momentum","method_name":"SGD with Momentum"},{"method_slug":"scheduleddroppath","method_name":"ScheduledDropPath"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[{"slug":"neural-architecture-search","name":"Neural Architecture Search","full_name":"Neural Architecture Search"}],"results":[{"leaderboard":"/sota/classification-on-indl","task":"Classification","dataset":"InDL","model":"NASNetLarge","rank_in_archive_order":6,"of":9,"metrics":{"Average Recall":"89.52%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"NASNET-A(6)","rank_in_archive_order":515,"of":1060,"metrics":{"GFLOPs":"23.8","Hardware Burden":"1648G","Number of params":"88.9M","Operations per network pass":"2.38G","Top 1 Accuracy":"82.7%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet-real","task":"Image Classification","dataset":"ImageNet ReaL","model":"NASNet-A Large","rank_in_archive_order":32,"of":57,"metrics":{"Accuracy":"87.56%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet-real","task":"Image Classification","dataset":"ImageNet ReaL","model":"NASNet-A Mobile","rank_in_archive_order":50,"of":57,"metrics":{"Accuracy":"81.15%"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1707.07012","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1707.07012"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tensorflow/models","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MarSaKi/nasnet","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tensorflow/models/tree/master/research/slim","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/johannesu/NASNet-keras","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mindspore-ai/models/tree/master/official/cv/nasnet","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/2024-MindSpore-1/Code6/tree/main/ArbitraryStyleTransfer","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/2024-MindSpore-1/Code5/tree/main/ArbitraryStyleTransfer","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/osmr/imgclsmob","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/titu1994/neural-architecture-search","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/asad-62/IVP-DNN","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tally0818/NASNet","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MindSpore-paper-code-3/code1/tree/main/ArbitraryStyleTransfer","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mindspore-ecosystem/mindcv/blob/main/mindcv/models/nasnet.py","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Tingelam/tianchi_gd_defect_pytorch_round1","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xufanxiong/classification-of-caltech-256","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mrzzy/np-dl-assign-1","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/DataCanvasIO/Hypernets","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":1},"by_repo_kind":{"listed":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"7e7f8c1742d5d0f1","entry":"Cell","repo":"MarSaKi/nasnet","repo_kind":"listed","path":"model.py","file_url":"https://github.com/MarSaKi/nasnet/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7e7f8c1742d5d0f1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}