{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mtl-nas-task-agnostic-neural-architecture","title":"MTL-NAS: Task-Agnostic Neural Architecture Search towards General-Purpose Multi-Task Learning","arxiv_id":"2003.14058","date":"2020-03-31","proceeding":"CVPR 2020 6","authors":["Yuan Gao","Haoping Bai","Zequn Jie","Jiayi Ma","Kui Jia","Wei Liu"],"abstract":"We propose to incorporate neural architecture search (NAS) into general-purpose multi-task learning (GP-MTL). Existing NAS methods typically define different search spaces according to different tasks. In order to adapt to different task combinations (i.e., task sets), we disentangle the GP-MTL networks into single-task backbones (optionally encode the task priors), and a hierarchical and layerwise features sharing/fusing scheme across them. This enables us to design a novel and general task-agnostic search space, which inserts cross-task edges (i.e., feature fusion connections) into fixed single-task network backbones. Moreover, we also propose a novel single-shot gradient-based search algorithm that closes the performance gap between the searched architectures and the final evaluation architecture. This is realized with a minimum entropy regularization on the architecture weights during the search phase, which makes the architecture weights converge to near-discrete values and therefore achieves a single model. As a result, our searched model can be directly used for evaluation without (re-)training from scratch. We perform extensive experiments using different single-task backbones on various task sets, demonstrating the promising performance obtained by exploiting the hierarchical and layerwise features, as well as the desirable generalizability to different i) task sets and ii) single-task backbones. The code of our paper is available at https://github.com/bhpfelix/MTLNAS.","url_abs":"https://arxiv.org/abs/2003.14058v1","url_pdf":"https://arxiv.org/pdf/2003.14058v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mtl-nas-task-agnostic-neural-architecture","repo_url":"https://github.com/bhpfelix/MTLNAS","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"multi-task-learning","task_name":"Multi-Task Learning"},{"task_slug":"architecture-search","task_name":"Neural Architecture Search"}],"methods":[{"method_slug":"entropy-regularization","method_name":"Entropy Regularization"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2003.14058","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2003.14058"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bhpfelix/MTLNAS","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":2,"unverified":11},"by_repo_kind":{"official":{"samples":13,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"56af5b6a5714675c","entry":"batch_norm","repo":"bhpfelix/MTLNAS","repo_kind":"official","path":"core/models/common_layers.py","file_url":"https://github.com/bhpfelix/MTLNAS/blob/HEAD/core/models/common_layers.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"56af5b6a5714675c"}},{"code_sha256_prefix":"da5bbab470adc6d7","entry":"poly","repo":"bhpfelix/MTLNAS","repo_kind":"official","path":"core/utils/losses.py","file_url":"https://github.com/bhpfelix/MTLNAS/blob/HEAD/core/utils/losses.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"da5bbab470adc6d7"}},{"code_sha256_prefix":"27f37e43a9a8bf2a","entry":"compute_angle","repo":"bhpfelix/MTLNAS","repo_kind":"official","path":"core/utils/metrics.py","file_url":"https://github.com/bhpfelix/MTLNAS/blob/HEAD/core/utils/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"27f37e43a9a8bf2a"}},{"code_sha256_prefix":"ca9ac840bc586a01","entry":"compute_hist","repo":"bhpfelix/MTLNAS","repo_kind":"official","path":"core/utils/metrics.py","file_url":"https://github.com/bhpfelix/MTLNAS/blob/HEAD/core/utils/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ca9ac840bc586a01"}},{"code_sha256_prefix":"be1d4c78c6570d4f","entry":"count_correct","repo":"bhpfelix/MTLNAS","repo_kind":"official","path":"core/utils/metrics.py","file_url":"https://github.com/bhpfelix/MTLNAS/blob/HEAD/core/utils/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"be1d4c78c6570d4f"}},{"code_sha256_prefix":"fc79b5b14193af1e","entry":"get_nddr","repo":"bhpfelix/MTLNAS","repo_kind":"official","path":"core/models/common_layers.py","file_url":"https://github.com/bhpfelix/MTLNAS/blob/HEAD/core/models/common_layers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fc79b5b14193af1e"}},{"code_sha256_prefix":"10b8ad22843e7d02","entry":"get_nddr_bn","repo":"bhpfelix/MTLNAS","repo_kind":"official","path":"core/models/common_layers.py","file_url":"https://github.com/bhpfelix/MTLNAS/blob/HEAD/core/models/common_layers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"10b8ad22843e7d02"}},{"code_sha256_prefix":"23e12e7e82ed9859","entry":"get_tasks","repo":"bhpfelix/MTLNAS","repo_kind":"official","path":"core/tasks.py","file_url":"https://github.com/bhpfelix/MTLNAS/blob/HEAD/core/tasks.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"23e12e7e82ed9859"}},{"code_sha256_prefix":"408334df60d6a716","entry":"normal_loss","repo":"bhpfelix/MTLNAS","repo_kind":"official","path":"core/utils/losses.py","file_url":"https://github.com/bhpfelix/MTLNAS/blob/HEAD/core/utils/losses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"408334df60d6a716"}},{"code_sha256_prefix":"a3dc17b381d3804c","entry":"normalize","repo":"bhpfelix/MTLNAS","repo_kind":"official","path":"core/utils/visualization.py","file_url":"https://github.com/bhpfelix/MTLNAS/blob/HEAD/core/utils/visualization.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a3dc17b381d3804c"}},{"code_sha256_prefix":"e85c27ee7db9da35","entry":"process_image","repo":"bhpfelix/MTLNAS","repo_kind":"official","path":"core/utils/visualization.py","file_url":"https://github.com/bhpfelix/MTLNAS/blob/HEAD/core/utils/visualization.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e85c27ee7db9da35"}},{"code_sha256_prefix":"40e5c245b7465f9b","entry":"process_seg_label","repo":"bhpfelix/MTLNAS","repo_kind":"official","path":"core/utils/visualization.py","file_url":"https://github.com/bhpfelix/MTLNAS/blob/HEAD/core/utils/visualization.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"40e5c245b7465f9b"}},{"code_sha256_prefix":"2bc85b26189ce633","entry":"seg_loss","repo":"bhpfelix/MTLNAS","repo_kind":"official","path":"core/utils/losses.py","file_url":"https://github.com/bhpfelix/MTLNAS/blob/HEAD/core/utils/losses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2bc85b26189ce633"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}