{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/self-supervised-representation-learning-for-3","title":"Self-supervised Representation Learning for Evolutionary Neural Architecture Search","arxiv_id":"2011.00186","date":"2020-10-31","proceeding":null,"authors":["Chen Wei","Yiping Tang","Chuang Niu","Haihong Hu","Yue Wang","Jimin Liang"],"abstract":"Recently proposed neural architecture search (NAS) algorithms adopt neural predictors to accelerate the architecture search. The capability of neural predictors to accurately predict the performance metrics of neural architecture is critical to NAS, and the acquisition of training datasets for neural predictors is time-consuming. How to obtain a neural predictor with high prediction accuracy using a small amount of training data is a central problem to neural predictor-based NAS. Here, we firstly design a new architecture encoding scheme that overcomes the drawbacks of existing vector-based architecture encoding schemes to calculate the graph edit distance of neural architectures. To enhance the predictive performance of neural predictors, we devise two self-supervised learning methods from different perspectives to pre-train the architecture embedding part of neural predictors to generate a meaningful representation of neural architectures. The first one is to train a carefully designed two branch graph neural network model to predict the graph edit distance of two input neural architectures. The second method is inspired by the prevalently contrastive learning, and we present a new contrastive learning algorithm that utilizes a central feature vector as a proxy to contrast positive pairs against negative pairs. Experimental results illustrate that the pre-trained neural predictors can achieve comparable or superior performance compared with their supervised counterparts with several times less training samples. We achieve state-of-the-art performance on the NASBench-101 and NASBench201 benchmarks when integrating the pre-trained neural predictors with an evolutionary NAS algorithm.","url_abs":"https://arxiv.org/abs/2011.00186v1","url_pdf":"https://arxiv.org/pdf/2011.00186v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"self-supervised-representation-learning-for-3","repo_url":"https://github.com/auroua/SSNENAS","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"graph-neural-network","task_name":"Graph Neural Network"},{"task_slug":"architecture-search","task_name":"Neural Architecture Search"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"},{"method_slug":"graph-neural-network","method_name":"Graph Neural Network"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2011.00186","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2011.00186"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/auroua/SSNENAS","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":13},"by_repo_kind":{"official":{"samples":13,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8e50b14abde79c1b","entry":"acq_fn","repo":"auroua/SSNENAS","repo_kind":"official","path":"nas_lib/algos_nas/acquisition_functions.py","file_url":"https://github.com/auroua/SSNENAS/blob/HEAD/nas_lib/algos_nas/acquisition_functions.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8e50b14abde79c1b"}},{"code_sha256_prefix":"96ee1ba824202f2b","entry":"algo_params_close_domain","repo":"auroua/SSNENAS","repo_kind":"official","path":"nas_lib/params_nas.py","file_url":"https://github.com/auroua/SSNENAS/blob/HEAD/nas_lib/params_nas.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"96ee1ba824202f2b"}},{"code_sha256_prefix":"c38b7508cdb1f030","entry":"algo_params_open_domain","repo":"auroua/SSNENAS","repo_kind":"official","path":"nas_lib/params_nas.py","file_url":"https://github.com/auroua/SSNENAS/blob/HEAD/nas_lib/params_nas.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c38b7508cdb1f030"}},{"code_sha256_prefix":"b86e1c4f11405bbc","entry":"build_open_algos","repo":"auroua/SSNENAS","repo_kind":"official","path":"nas_lib/algos_darts/build_open_algos.py","file_url":"https://github.com/auroua/SSNENAS/blob/HEAD/nas_lib/algos_darts/build_open_algos.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b86e1c4f11405bbc"}},{"code_sha256_prefix":"cb94f7ba1f7afae8","entry":"compute_best_test_losses","repo":"auroua/SSNENAS","repo_kind":"official","path":"nas_lib/algos_nas/algo_compare.py","file_url":"https://github.com/auroua/SSNENAS/blob/HEAD/nas_lib/algos_nas/algo_compare.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cb94f7ba1f7afae8"}},{"code_sha256_prefix":"7f5760af9d9e1fdb","entry":"evolution_search_nasbench_101","repo":"auroua/SSNENAS","repo_kind":"official","path":"nas_lib/algos_nas/evolution.py","file_url":"https://github.com/auroua/SSNENAS/blob/HEAD/nas_lib/algos_nas/evolution.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7f5760af9d9e1fdb"}},{"code_sha256_prefix":"fd6439b6def33330","entry":"meta_neuralnet_params","repo":"auroua/SSNENAS","repo_kind":"official","path":"nas_lib/params_nas.py","file_url":"https://github.com/auroua/SSNENAS/blob/HEAD/nas_lib/params_nas.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fd6439b6def33330"}},{"code_sha256_prefix":"383278a1ef55d3cd","entry":"quick_sort_list","repo":"auroua/SSNENAS","repo_kind":"official","path":"nas_lib/algos/algo_sort.py","file_url":"https://github.com/auroua/SSNENAS/blob/HEAD/nas_lib/algos/algo_sort.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"383278a1ef55d3cd"}},{"code_sha256_prefix":"d9cc4398d1baf37a","entry":"run_nas_algos_nasbench_101","repo":"auroua/SSNENAS","repo_kind":"official","path":"nas_lib/algos_nas/algo_compare.py","file_url":"https://github.com/auroua/SSNENAS/blob/HEAD/nas_lib/algos_nas/algo_compare.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d9cc4398d1baf37a"}},{"code_sha256_prefix":"d6d169f03ec8c6ea","entry":"run_nas_algos_nasbench_201","repo":"auroua/SSNENAS","repo_kind":"official","path":"nas_lib/algos_nas/algo_compare.py","file_url":"https://github.com/auroua/SSNENAS/blob/HEAD/nas_lib/algos_nas/algo_compare.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d6d169f03ec8c6ea"}},{"code_sha256_prefix":"33da0703810ce533","entry":"supervised_ss_rl_params","repo":"auroua/SSNENAS","repo_kind":"official","path":"nas_lib/params.py","file_url":"https://github.com/auroua/SSNENAS/blob/HEAD/nas_lib/params.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"33da0703810ce533"}},{"code_sha256_prefix":"4aa9f462c2fbfbcb","entry":"unsupervised_ss_rl_params","repo":"auroua/SSNENAS","repo_kind":"official","path":"nas_lib/params.py","file_url":"https://github.com/auroua/SSNENAS/blob/HEAD/nas_lib/params.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4aa9f462c2fbfbcb"}},{"code_sha256_prefix":"ac8329f01f864a9e","entry":"unsupervised_ss_rl_pre_train","repo":"auroua/SSNENAS","repo_kind":"official","path":"nas_lib/params.py","file_url":"https://github.com/auroua/SSNENAS/blob/HEAD/nas_lib/params.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ac8329f01f864a9e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}