{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/neural-architecture-search-for-lightweight","title":"Neural Architecture Search for Lightweight Non-Local Networks","arxiv_id":"2004.01961","date":"2020-04-04","proceeding":"CVPR 2020 6","authors":["Yingwei Li","Xiaojie Jin","Jieru Mei","Xiaochen Lian","Linjie Yang","Cihang Xie","Qihang Yu","Yuyin Zhou","Song Bai","Alan Yuille"],"abstract":"Non-Local (NL) blocks have been widely studied in various vision tasks. However, it has been rarely explored to embed the NL blocks in mobile neural networks, mainly due to the following challenges: 1) NL blocks generally have heavy computation cost which makes it difficult to be applied in applications where computational resources are limited, and 2) it is an open problem to discover an optimal configuration to embed NL blocks into mobile neural networks. We propose AutoNL to overcome the above two obstacles. Firstly, we propose a Lightweight Non-Local (LightNL) block by squeezing the transformation operations and incorporating compact features. With the novel design choices, the proposed LightNL block is 400x computationally cheaper} than its conventional counterpart without sacrificing the performance. Secondly, by relaxing the structure of the LightNL block to be differentiable during training, we propose an efficient neural architecture search algorithm to learn an optimal configuration of LightNL blocks in an end-to-end manner. Notably, using only 32 GPU hours, the searched AutoNL model achieves 77.7% top-1 accuracy on ImageNet under a typical mobile setting (350M FLOPs), significantly outperforming previous mobile models including MobileNetV2 (+5.7%), FBNet (+2.8%) and MnasNet (+2.1%). Code and models are available at https://github.com/LiYingwei/AutoNL.","url_abs":"https://arxiv.org/abs/2004.01961v1","url_pdf":"https://arxiv.org/pdf/2004.01961v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"neural-architecture-search-for-lightweight","repo_url":"https://github.com/LiYingwei/AutoNL","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"neural-architecture-search-for-lightweight","repo_url":"https://github.com/meijieru/yet_another_mobilenet_series","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":null,"task_name":"GPU"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"architecture-search","task_name":"Neural Architecture Search"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"depthwise-convolution","method_name":"Depthwise Convolution"},{"method_slug":"depthwise-separable-convolution","method_name":"Depthwise Separable Convolution"},{"method_slug":"grouped-convolution","method_name":"Grouped Convolution"},{"method_slug":"inverted-residual-block","method_name":"Inverted Residual Block"},{"method_slug":"pointwise-convolution","method_name":"Pointwise Convolution"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/neural-architecture-search-on-imagenet","task":"Neural Architecture Search","dataset":"ImageNet","model":"AutoNL-L","rank_in_archive_order":60,"of":135,"metrics":{"Accuracy":"77.7","Top-1 Error Rate":"22.3"},"uses_additional_data":false},{"leaderboard":"/sota/neural-architecture-search-on-imagenet","task":"Neural Architecture Search","dataset":"ImageNet","model":"AutoNL-S","rank_in_archive_order":84,"of":135,"metrics":{"Accuracy":"76.5","Top-1 Error Rate":"23.5"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2004.01961","atlas_url":"https://app.syntology.ai/?focus=2004.01961","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2004.01961"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/LiYingwei/AutoNL","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/meijieru/yet_another_mobilenet_series","reach":null}],"summary":{"ran_honours":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"285e4d1d9a3faeba","entry":"round_filters","repo":"LiYingwei/AutoNL","repo_kind":"official","path":"model_def.py","file_url":"https://github.com/LiYingwei/AutoNL/blob/HEAD/model_def.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"285e4d1d9a3faeba"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}