{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rnas-architecture-ranking-for-powerful","title":"ReNAS:Relativistic Evaluation of Neural Architecture Search","arxiv_id":"1910.01523","date":"2019-09-30","proceeding":null,"authors":["Yixing Xu","Yunhe Wang","Kai Han","Yehui Tang","Shangling Jui","Chunjing Xu","Chang Xu"],"abstract":"An effective and efficient architecture performance evaluation scheme is essential for the success of Neural Architecture Search (NAS). To save computational cost, most of existing NAS algorithms often train and evaluate intermediate neural architectures on a small proxy dataset with limited training epochs. But it is difficult to expect an accurate performance estimation of an architecture in such a coarse evaluation way. This paper advocates a new neural architecture evaluation scheme, which aims to determine which architecture would perform better instead of accurately predict the absolute architecture performance. Therefore, we propose a \\textbf{relativistic} architecture performance predictor in NAS (ReNAS). We encode neural architectures into feature tensors, and further refining the representations with the predictor. The proposed relativistic performance predictor can be deployed in discrete searching methods to search for the desired architectures without additional evaluation. Experimental results on NAS-Bench-101 dataset suggests that, sampling 424 ($0.1\\%$ of the entire search space) neural architectures and their corresponding validation performance is already enough for learning an accurate architecture performance predictor. The accuracies of our searched neural architectures on NAS-Bench-101 and NAS-Bench-201 datasets are higher than that of the state-of-the-art methods and show the priority of the proposed method.","url_abs":"https://arxiv.org/abs/1910.01523v7","url_pdf":"https://arxiv.org/pdf/1910.01523v7.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rnas-architecture-ranking-for-powerful","repo_url":"https://github.com/huawei-noah/Efficient-Computing","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"rnas-architecture-ranking-for-powerful","repo_url":"https://github.com/albertipot/nar","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"rnas-architecture-ranking-for-powerful","repo_url":"https://github.com/mindspore-ai/models/tree/master/research/cv/renas","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"rnas-architecture-ranking-for-powerful","repo_url":"https://github.com/xiaoshingshing/ReNAS","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"architecture-search","task_name":"Neural Architecture Search"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1910.01523","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1910.01523"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mindspore-ai/models/tree/master/research/cv/renas","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xiaoshingshing/ReNAS","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/huawei-noah/Efficient-Computing","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/albertipot/nar","reach":null}],"summary":{"ran_draft_wrong":3,"ran_fixture":1,"unverified":2},"by_repo_kind":{"listed":{"samples":6,"ran":4,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"901fcd2b10d137c7","entry":"adjacency_matrix_padding","repo":"albertipot/nar","repo_kind":"listed","path":"architecture/arch_encode.py","file_url":"https://github.com/albertipot/nar/blob/HEAD/architecture/arch_encode.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"901fcd2b10d137c7"}},{"code_sha256_prefix":"a9680e2d00e1eca3","entry":"get_dataset","repo":"xiaoshingshing/ReNAS","repo_kind":"listed","path":"renas.py","file_url":"https://github.com/xiaoshingshing/ReNAS/blob/HEAD/renas.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"a9680e2d00e1eca3"}},{"code_sha256_prefix":"993015b25d40d382","entry":"right_pair","repo":"xiaoshingshing/ReNAS","repo_kind":"listed","path":"renas.py","file_url":"https://github.com/xiaoshingshing/ReNAS/blob/HEAD/renas.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"993015b25d40d382"}},{"code_sha256_prefix":"4c8687589d3c4e24","entry":"vector_padding","repo":"albertipot/nar","repo_kind":"listed","path":"architecture/arch_encode.py","file_url":"https://github.com/albertipot/nar/blob/HEAD/architecture/arch_encode.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"4c8687589d3c4e24"}},{"code_sha256_prefix":"5155bf6598485e16","entry":"feature_tensor_encoding","repo":"albertipot/nar","repo_kind":"listed","path":"architecture/arch_encode.py","file_url":"https://github.com/albertipot/nar/blob/HEAD/architecture/arch_encode.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"5155bf6598485e16"}},{"code_sha256_prefix":"e76cf548cfcc2ebf","entry":"get_data_from_file","repo":"xiaoshingshing/ReNAS","repo_kind":"listed","path":"renas.py","file_url":"https://github.com/xiaoshingshing/ReNAS/blob/HEAD/renas.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"e76cf548cfcc2ebf"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}