{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/hr-nas-searching-efficient-high-resolution","title":"HR-NAS: Searching Efficient High-Resolution Neural Architectures with Lightweight Transformers","arxiv_id":"2106.06560","date":"2021-06-11","proceeding":"CVPR 2021 1","authors":["Mingyu Ding","Xiaochen Lian","Linjie Yang","Peng Wang","Xiaojie Jin","Zhiwu Lu","Ping Luo"],"abstract":"High-resolution representations (HR) are essential for dense prediction tasks such as segmentation, detection, and pose estimation. Learning HR representations is typically ignored in previous Neural Architecture Search (NAS) methods that focus on image classification. This work proposes a novel NAS method, called HR-NAS, which is able to find efficient and accurate networks for different tasks, by effectively encoding multiscale contextual information while maintaining high-resolution representations. In HR-NAS, we renovate the NAS search space as well as its searching strategy. To better encode multiscale image contexts in the search space of HR-NAS, we first carefully design a lightweight transformer, whose computational complexity can be dynamically changed with respect to different objective functions and computation budgets. To maintain high-resolution representations of the learned networks, HR-NAS adopts a multi-branch architecture that provides convolutional encoding of multiple feature resolutions, inspired by HRNet. Last, we proposed an efficient fine-grained search strategy to train HR-NAS, which effectively explores the search space, and finds optimal architectures given various tasks and computation resources. HR-NAS is capable of achieving state-of-the-art trade-offs between performance and FLOPs for three dense prediction tasks and an image classification task, given only small computational budgets. For example, HR-NAS surpasses SqueezeNAS that is specially designed for semantic segmentation while improving efficiency by 45.9%. Code is available at https://github.com/dingmyu/HR-NAS","url_abs":"https://arxiv.org/abs/2106.06560v1","url_pdf":"https://arxiv.org/pdf/2106.06560v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"hr-nas-searching-efficient-high-resolution","repo_url":"https://github.com/dingmyu/HR-NAS","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"architecture-search","task_name":"Neural Architecture Search"},{"task_slug":"pose-estimation","task_name":"Pose Estimation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"high","task_name":"Vocal Bursts Intensity Prediction"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"hrnet","method_name":"HRNet"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-connection","method_name":"Residual Connection"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2106.06560","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.06560"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/dingmyu/HR-NAS","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"ran":2,"ran_fixture":1,"unverified":8},"by_repo_kind":{"official":{"samples":12,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"fac5364e2f53c6db","entry":"conv3x3","repo":"dingmyu/HR-NAS","repo_kind":"official","path":"models/hrnet_base.py","file_url":"https://github.com/dingmyu/HR-NAS/blob/HEAD/models/hrnet_base.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fac5364e2f53c6db"}},{"code_sha256_prefix":"32f1f27f7d19f6ad","entry":"conv_module_name_filter","repo":"dingmyu/HR-NAS","repo_kind":"official","path":"utils/model_profiling.py","file_url":"https://github.com/dingmyu/HR-NAS/blob/HEAD/utils/model_profiling.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"32f1f27f7d19f6ad"}},{"code_sha256_prefix":"e2db4aa1fd031021","entry":"get_params","repo":"dingmyu/HR-NAS","repo_kind":"official","path":"utils/model_profiling.py","file_url":"https://github.com/dingmyu/HR-NAS/blob/HEAD/utils/model_profiling.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e2db4aa1fd031021"}},{"code_sha256_prefix":"e8fbff5c3395ac23","entry":"resize","repo":"dingmyu/HR-NAS","repo_kind":"official","path":"models/transformer.py","file_url":"https://github.com/dingmyu/HR-NAS/blob/HEAD/models/transformer.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e8fbff5c3395ac23"}},{"code_sha256_prefix":"b62c4cc79bbeaff7","entry":"accuracy","repo":"dingmyu/HR-NAS","repo_kind":"official","path":"mmseg/accuracy.py","file_url":"https://github.com/dingmyu/HR-NAS/blob/HEAD/mmseg/accuracy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b62c4cc79bbeaff7"}},{"code_sha256_prefix":"386bec0c5e132dd1","entry":"binary_cross_entropy","repo":"dingmyu/HR-NAS","repo_kind":"official","path":"mmseg/loss.py","file_url":"https://github.com/dingmyu/HR-NAS/blob/HEAD/mmseg/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"386bec0c5e132dd1"}},{"code_sha256_prefix":"156683f98b5dd1b4","entry":"cross_entropy","repo":"dingmyu/HR-NAS","repo_kind":"official","path":"mmseg/loss.py","file_url":"https://github.com/dingmyu/HR-NAS/blob/HEAD/mmseg/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"156683f98b5dd1b4"}},{"code_sha256_prefix":"f344ad03a3e1b61b","entry":"get_active_fn","repo":"dingmyu/HR-NAS","repo_kind":"official","path":"models/mobilenet_base.py","file_url":"https://github.com/dingmyu/HR-NAS/blob/HEAD/models/mobilenet_base.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f344ad03a3e1b61b"}},{"code_sha256_prefix":"954418d7666daede","entry":"get_points_single","repo":"dingmyu/HR-NAS","repo_kind":"official","path":"models/transformer.py","file_url":"https://github.com/dingmyu/HR-NAS/blob/HEAD/models/transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"954418d7666daede"}},{"code_sha256_prefix":"13224d835509143a","entry":"mask_cross_entropy","repo":"dingmyu/HR-NAS","repo_kind":"official","path":"mmseg/loss.py","file_url":"https://github.com/dingmyu/HR-NAS/blob/HEAD/mmseg/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"13224d835509143a"}},{"code_sha256_prefix":"39bddff1555b4a0c","entry":"output_network","repo":"dingmyu/HR-NAS","repo_kind":"official","path":"models/mobilenet_base.py","file_url":"https://github.com/dingmyu/HR-NAS/blob/HEAD/models/mobilenet_base.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"39bddff1555b4a0c"}},{"code_sha256_prefix":"17c195449846c9ea","entry":"run_forward","repo":"dingmyu/HR-NAS","repo_kind":"official","path":"utils/model_profiling.py","file_url":"https://github.com/dingmyu/HR-NAS/blob/HEAD/utils/model_profiling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"17c195449846c9ea"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}