{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/aux-nas-exploiting-auxiliary-labels-with","title":"Aux-NAS: Exploiting Auxiliary Labels with Negligibly Extra Inference Cost","arxiv_id":"2405.05695","date":"2024-05-09","proceeding":null,"authors":["Yuan Gao","Weizhong Zhang","Wenhan Luo","Lin Ma","Jin-Gang Yu","Gui-Song Xia","Jiayi Ma"],"abstract":"We aim at exploiting additional auxiliary labels from an independent (auxiliary) task to boost the primary task performance which we focus on, while preserving a single task inference cost of the primary task. While most existing auxiliary learning methods are optimization-based relying on loss weights/gradients manipulation, our method is architecture-based with a flexible asymmetric structure for the primary and auxiliary tasks, which produces different networks for training and inference. Specifically, starting from two single task networks/branches (each representing a task), we propose a novel method with evolving networks where only primary-to-auxiliary links exist as the cross-task connections after convergence. These connections can be removed during the primary task inference, resulting in a single-task inference cost. We achieve this by formulating a Neural Architecture Search (NAS) problem, where we initialize bi-directional connections in the search space and guide the NAS optimization converging to an architecture with only the single-side primary-to-auxiliary connections. Moreover, our method can be incorporated with optimization-based auxiliary learning approaches. Extensive experiments with six tasks on NYU v2, CityScapes, and Taskonomy datasets using VGG, ResNet, and ViT backbones validate the promising performance. The codes are available at https://github.com/ethanygao/Aux-NAS.","url_abs":"https://arxiv.org/abs/2405.05695v1","url_pdf":"https://arxiv.org/pdf/2405.05695v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"aux-nas-exploiting-auxiliary-labels-with","repo_url":"https://github.com/ethanygao/aux-nas","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"auxiliary-learning","task_name":"Auxiliary Learning"},{"task_slug":"architecture-search","task_name":"Neural Architecture Search"}],"methods":[{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"focus","method_name":"Focus"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.05695","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.05695"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/ethanygao/Aux-NAS","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ethanygao/aux-nas","reach":{"status":"ok"}}],"summary":{"ran":7,"ran_draft_wrong":5,"ran_honours":1,"unverified":2},"by_repo_kind":{"official":{"samples":15,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":15,"samples":[{"code_sha256_prefix":"cc1d6490e45a9d27","entry":"SingleSidedAsymmetricResidualNDDR","repo":"ethanygao/aux-nas","repo_kind":"official","path":"networks/stagewise_search_aux.py","file_url":"https://github.com/ethanygao/aux-nas/blob/HEAD/networks/stagewise_search_aux.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cc1d6490e45a9d27"}},{"code_sha256_prefix":"56af5b6a5714675c","entry":"batch_norm","repo":"ethanygao/aux-nas","repo_kind":"official","path":"networks/stagewise_search_aux.py","file_url":"https://github.com/ethanygao/aux-nas/blob/HEAD/networks/stagewise_search_aux.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"56af5b6a5714675c"}},{"code_sha256_prefix":"25ba9b5031480973","entry":"compute_scale_and_shift","repo":"ethanygao/Aux-NAS","repo_kind":"official","path":"loss/losses.py","file_url":"https://github.com/ethanygao/Aux-NAS/blob/HEAD/loss/losses.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"25ba9b5031480973"}},{"code_sha256_prefix":"7219923056d8addc","entry":"depth_limited_connectivity_matrix","repo":"ethanygao/Aux-NAS","repo_kind":"official","path":"networks/aux_model.py","file_url":"https://github.com/ethanygao/Aux-NAS/blob/HEAD/networks/aux_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7219923056d8addc"}},{"code_sha256_prefix":"b2ce8e695ddf3a7c","entry":"entropy_loss","repo":"ethanygao/aux-nas","repo_kind":"official","path":"networks/stagewise_search_aux.py","file_url":"https://github.com/ethanygao/aux-nas/blob/HEAD/networks/stagewise_search_aux.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b2ce8e695ddf3a7c"}},{"code_sha256_prefix":"8346d5d38e5f0042","entry":"get_logger","repo":"ethanygao/Aux-NAS","repo_kind":"official","path":"utils/logger.py","file_url":"https://github.com/ethanygao/Aux-NAS/blob/HEAD/utils/logger.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8346d5d38e5f0042"}},{"code_sha256_prefix":"b1af115958c103bc","entry":"get_nddr","repo":"ethanygao/aux-nas","repo_kind":"official","path":"networks/stagewise_search_aux.py","file_url":"https://github.com/ethanygao/aux-nas/blob/HEAD/networks/stagewise_search_aux.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b1af115958c103bc"}},{"code_sha256_prefix":"4ab0e3296c5e44f2","entry":"get_optimizer_aux","repo":"ethanygao/Aux-NAS","repo_kind":"official","path":"utils/optim.py","file_url":"https://github.com/ethanygao/Aux-NAS/blob/HEAD/utils/optim.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4ab0e3296c5e44f2"}},{"code_sha256_prefix":"7b83f79fccb2b76d","entry":"get_optimizer_single","repo":"ethanygao/Aux-NAS","repo_kind":"official","path":"utils/optim.py","file_url":"https://github.com/ethanygao/Aux-NAS/blob/HEAD/utils/optim.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7b83f79fccb2b76d"}},{"code_sha256_prefix":"f8b84bd1ce7b7e94","entry":"l1_loss","repo":"ethanygao/aux-nas","repo_kind":"official","path":"networks/stagewise_search_aux.py","file_url":"https://github.com/ethanygao/aux-nas/blob/HEAD/networks/stagewise_search_aux.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f8b84bd1ce7b7e94"}},{"code_sha256_prefix":"ddb859dc4e680704","entry":"layer_norm","repo":"ethanygao/aux-nas","repo_kind":"official","path":"networks/stagewise_search_aux.py","file_url":"https://github.com/ethanygao/aux-nas/blob/HEAD/networks/stagewise_search_aux.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ddb859dc4e680704"}},{"code_sha256_prefix":"da5bbab470adc6d7","entry":"poly","repo":"ethanygao/aux-nas","repo_kind":"official","path":"networks/stagewise_search_aux.py","file_url":"https://github.com/ethanygao/aux-nas/blob/HEAD/networks/stagewise_search_aux.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"da5bbab470adc6d7"}},{"code_sha256_prefix":"e8996630fe3501e1","entry":"reduction_batch_based","repo":"ethanygao/Aux-NAS","repo_kind":"official","path":"loss/losses.py","file_url":"https://github.com/ethanygao/Aux-NAS/blob/HEAD/loss/losses.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e8996630fe3501e1"}},{"code_sha256_prefix":"dbdcd5113506addd","entry":"GeneralizedMTLNASNet_L1Cut","repo":"ethanygao/aux-nas","repo_kind":"official","path":"networks/stagewise_search_aux.py","file_url":"https://github.com/ethanygao/aux-nas/blob/HEAD/networks/stagewise_search_aux.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dbdcd5113506addd"}},{"code_sha256_prefix":"c92f0115a72bf454","entry":"reduction_image_based","repo":"ethanygao/Aux-NAS","repo_kind":"official","path":"loss/losses.py","file_url":"https://github.com/ethanygao/Aux-NAS/blob/HEAD/loss/losses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c92f0115a72bf454"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}