{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multi-task-structural-learning-using-local","title":"Multi-Task Structural Learning using Local Task Similarity induced Neuron Creation and Removal","arxiv_id":"2305.00441","date":"2023-04-30","proceeding":null,"authors":["Naresh Kumar Gurulingan","Bahram Zonooz","Elahe Arani"],"abstract":"Multi-task learning has the potential to improve generalization by maximizing positive transfer between tasks while reducing task interference. Fully achieving this potential is hindered by manually designed architectures that remain static throughout training. On the contrary, learning in the brain occurs through structural changes that are in tandem with changes in synaptic strength. Thus, we propose \\textit{Multi-Task Structural Learning (MTSL)} that simultaneously learns the multi-task architecture and its parameters. MTSL begins with an identical single-task network for each task and alternates between a task-learning phase and a structural-learning phase. In the task learning phase, each network specializes in the corresponding task. In each of the structural learning phases, starting from the earliest layer, locally similar task layers first transfer their knowledge to a newly created group layer before being removed. MTSL then uses the group layer in place of the corresponding removed task layers and moves on to the next layers. Our empirical results show that MTSL achieves competitive generalization with various baselines and improves robustness to out-of-distribution data.","url_abs":"https://arxiv.org/abs/2305.00441v1","url_pdf":"https://arxiv.org/pdf/2305.00441v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multi-task-structural-learning-using-local","repo_url":"https://github.com/neurai-lab/mtsl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"multi-task-learning","task_name":"Multi-Task Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.00441","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.00441"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/NeurAI-Lab/MTSL","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/neurai-lab/mtsl","reach":{"status":"ok"}}],"summary":{"ran":2,"unverified":3},"by_repo_kind":{"official":{"samples":5,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"66ad2f9cab83b661","entry":"ShatteredStages","repo":"neurai-lab/mtsl","repo_kind":"official","path":"encoding_custom/models/mtsl.py","file_url":"https://github.com/neurai-lab/mtsl/blob/HEAD/encoding_custom/models/mtsl.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"66ad2f9cab83b661"}},{"code_sha256_prefix":"4feeaf9d6a010187","entry":"TaskHead","repo":"neurai-lab/mtsl","repo_kind":"official","path":"encoding_custom/models/mtsl.py","file_url":"https://github.com/neurai-lab/mtsl/blob/HEAD/encoding_custom/models/mtsl.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4feeaf9d6a010187"}},{"code_sha256_prefix":"5e36d66e8e4c4fb9","entry":"BaseNet","repo":"neurai-lab/mtsl","repo_kind":"official","path":"encoding_custom/models/mtsl.py","file_url":"https://github.com/neurai-lab/mtsl/blob/HEAD/encoding_custom/models/mtsl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5e36d66e8e4c4fb9"}},{"code_sha256_prefix":"4a118d215925736b","entry":"MTSLArch","repo":"neurai-lab/mtsl","repo_kind":"official","path":"encoding_custom/models/mtsl.py","file_url":"https://github.com/neurai-lab/mtsl/blob/HEAD/encoding_custom/models/mtsl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4a118d215925736b"}},{"code_sha256_prefix":"bfb7022a91e5720f","entry":"ModelAttributes","repo":"neurai-lab/mtsl","repo_kind":"official","path":"encoding_custom/models/mtsl.py","file_url":"https://github.com/neurai-lab/mtsl/blob/HEAD/encoding_custom/models/mtsl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bfb7022a91e5720f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}