{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/variational-multi-task-learning-with-gumbel","title":"Variational Multi-Task Learning with Gumbel-Softmax Priors","arxiv_id":"2111.05323","date":"2021-11-09","proceeding":"NeurIPS 2021 12","authors":["Jiayi Shen","XianTong Zhen","Marcel Worring","Ling Shao"],"abstract":"Multi-task learning aims to explore task relatedness to improve individual tasks, which is of particular significance in the challenging scenario that only limited data is available for each task. To tackle this challenge, we propose variational multi-task learning (VMTL), a general probabilistic inference framework for learning multiple related tasks. We cast multi-task learning as a variational Bayesian inference problem, in which task relatedness is explored in a unified manner by specifying priors. To incorporate shared knowledge into each task, we design the prior of a task to be a learnable mixture of the variational posteriors of other related tasks, which is learned by the Gumbel-Softmax technique. In contrast to previous methods, our VMTL can exploit task relatedness for both representations and classifiers in a principled way by jointly inferring their posteriors. This enables individual tasks to fully leverage inductive biases provided by related tasks, therefore improving the overall performance of all tasks. Experimental results demonstrate that the proposed VMTL is able to effectively tackle a variety of challenging multi-task learning settings with limited training data for both classification and regression. Our method consistently surpasses previous methods, including strong Bayesian approaches, and achieves state-of-the-art performance on five benchmark datasets.","url_abs":"https://arxiv.org/abs/2111.05323v1","url_pdf":"https://arxiv.org/pdf/2111.05323v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"variational-multi-task-learning-with-gumbel","repo_url":"https://github.com/autumn9999/vmtl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"bayesian-inference","task_name":"Bayesian Inference"},{"task_slug":"multi-task-learning","task_name":"Multi-Task Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2111.05323","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2111.05323"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/autumn9999/VMTL","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/thuml/MTlearn","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/autumn9999/vmtl","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":4,"ran":3,"unverified":6},"by_repo_kind":{"official":{"samples":8,"ran":6,"repositories":1},"found_in_text":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":13,"samples":[{"code_sha256_prefix":"ad6724b21272b7b3","entry":"gumbel_softmax","repo":"autumn9999/vmtl","repo_kind":"official","path":"model_vmtl.py","file_url":"https://github.com/autumn9999/vmtl/blob/HEAD/model_vmtl.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ad6724b21272b7b3"}},{"code_sha256_prefix":"09ef8173e6e07e61","entry":"gumbel_softmax_sample","repo":"autumn9999/vmtl","repo_kind":"official","path":"model_vmtl.py","file_url":"https://github.com/autumn9999/vmtl/blob/HEAD/model_vmtl.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"09ef8173e6e07e61"}},{"code_sha256_prefix":"8035cf57b4b550f8","entry":"kl_criterion_softplus","repo":"autumn9999/vmtl","repo_kind":"official","path":"model_vmtl.py","file_url":"https://github.com/autumn9999/vmtl/blob/HEAD/model_vmtl.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8035cf57b4b550f8"}},{"code_sha256_prefix":"0181920f28840de9","entry":"sample_gumbel","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"0181920f28840de9"}},{"code_sha256_prefix":"ff33f4241dfadae5","entry":"task_shared_network","repo":"autumn9999/vmtl","repo_kind":"official","path":"model_vmtl.py","file_url":"https://github.com/autumn9999/vmtl/blob/HEAD/model_vmtl.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ff33f4241dfadae5"}},{"code_sha256_prefix":"e12a18df21af23b1","entry":"task_specific_gumbel","repo":"autumn9999/vmtl","repo_kind":"official","path":"model_vmtl.py","file_url":"https://github.com/autumn9999/vmtl/blob/HEAD/model_vmtl.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e12a18df21af23b1"}},{"code_sha256_prefix":"8aaa2924dd7f7bb9","entry":"task_specific_network","repo":"autumn9999/vmtl","repo_kind":"official","path":"model_vmtl.py","file_url":"https://github.com/autumn9999/vmtl/blob/HEAD/model_vmtl.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8aaa2924dd7f7bb9"}},{"code_sha256_prefix":"c1b96192d2a78ef9","entry":"HomoMultiTaskModel","repo":"thuml/MTlearn","repo_kind":"found_in_text","path":"src/model_multi_task.py","file_url":"https://github.com/thuml/MTlearn/blob/HEAD/src/model_multi_task.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c1b96192d2a78ef9"}},{"code_sha256_prefix":"0b0cb97dda9d4228","entry":"MainModel","repo":"autumn9999/vmtl","repo_kind":"official","path":"model_vmtl.py","file_url":"https://github.com/autumn9999/vmtl/blob/HEAD/model_vmtl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0b0cb97dda9d4228"}},{"code_sha256_prefix":"e113c85acb13c2b3","entry":"MultiTaskLoss","repo":"thuml/MTlearn","repo_kind":"found_in_text","path":"src/model_multi_task.py","file_url":"https://github.com/thuml/MTlearn/blob/HEAD/src/model_multi_task.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e113c85acb13c2b3"}},{"code_sha256_prefix":"36a790c729e85178","entry":"UpdateCov","repo":"thuml/MTlearn","repo_kind":"found_in_text","path":"src/model_multi_task.py","file_url":"https://github.com/thuml/MTlearn/blob/HEAD/src/model_multi_task.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"36a790c729e85178"}},{"code_sha256_prefix":"c214973cf8590ca7","entry":"Vgg16NoFc","repo":"thuml/MTlearn","repo_kind":"found_in_text","path":"src/model_multi_task.py","file_url":"https://github.com/thuml/MTlearn/blob/HEAD/src/model_multi_task.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c214973cf8590ca7"}},{"code_sha256_prefix":"6c374d3a6b0b7997","entry":"local_reparameterize_softplus","repo":"autumn9999/vmtl","repo_kind":"official","path":"model_vmtl.py","file_url":"https://github.com/autumn9999/vmtl/blob/HEAD/model_vmtl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6c374d3a6b0b7997"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}