{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/synergies-between-disentanglement-and","title":"Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning","arxiv_id":"2211.14666","date":"2022-11-26","proceeding":null,"authors":["Sébastien Lachapelle","Tristan Deleu","Divyat Mahajan","Ioannis Mitliagkas","Yoshua Bengio","Simon Lacoste-Julien","Quentin Bertrand"],"abstract":"Although disentangled representations are often said to be beneficial for downstream tasks, current empirical and theoretical understanding is limited. In this work, we provide evidence that disentangled representations coupled with sparse base-predictors improve generalization. In the context of multi-task learning, we prove a new identifiability result that provides conditions under which maximally sparse base-predictors yield disentangled representations. Motivated by this theoretical result, we propose a practical approach to learn disentangled representations based on a sparsity-promoting bi-level optimization problem. Finally, we explore a meta-learning version of this algorithm based on group Lasso multiclass SVM base-predictors, for which we derive a tractable dual formulation. It obtains competitive results on standard few-shot classification benchmarks, while each task is using only a fraction of the learned representations.","url_abs":"https://arxiv.org/abs/2211.14666v2","url_pdf":"https://arxiv.org/pdf/2211.14666v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"synergies-between-disentanglement-and","repo_url":"https://github.com/tristandeleu/synergies-disentanglement-sparsity","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"disentanglement","task_name":"Disentanglement"},{"task_slug":"meta-learning","task_name":"Meta-Learning"},{"task_slug":"multi-task-learning","task_name":"Multi-Task Learning"}],"methods":[{"method_slug":"svm","method_name":"SVM"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2211.14666","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2211.14666"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tristandeleu/synergies-disentanglement-sparsity","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"ran_draft_wrong":1,"unverified":11},"by_repo_kind":{"official":{"samples":14,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"691b2fde39c5dae5","entry":"computeGramMatrix","repo":"tristandeleu/synergies-disentanglement-sparsity","repo_kind":"official","path":"metaoptnet/models/classification_heads.py","file_url":"https://github.com/tristandeleu/synergies-disentanglement-sparsity/blob/HEAD/metaoptnet/models/classification_heads.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"691b2fde39c5dae5"}},{"code_sha256_prefix":"fac5364e2f53c6db","entry":"conv3x3","repo":"tristandeleu/synergies-disentanglement-sparsity","repo_kind":"official","path":"metaoptnet/models/ResNet12_embedding.py","file_url":"https://github.com/tristandeleu/synergies-disentanglement-sparsity/blob/HEAD/metaoptnet/models/ResNet12_embedding.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fac5364e2f53c6db"}},{"code_sha256_prefix":"82506bf89d625519","entry":"index_classes","repo":"tristandeleu/synergies-disentanglement-sparsity","repo_kind":"official","path":"metaoptnet/omniglot_loaders.py","file_url":"https://github.com/tristandeleu/synergies-disentanglement-sparsity/blob/HEAD/metaoptnet/omniglot_loaders.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"82506bf89d625519"}},{"code_sha256_prefix":"cc80669c5bb2379d","entry":"R2D2_conv_block","repo":"tristandeleu/synergies-disentanglement-sparsity","repo_kind":"official","path":"metaoptnet/models/R2D2_embedding.py","file_url":"https://github.com/tristandeleu/synergies-disentanglement-sparsity/blob/HEAD/metaoptnet/models/R2D2_embedding.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cc80669c5bb2379d"}},{"code_sha256_prefix":"7d8d074dc5077026","entry":"binv","repo":"tristandeleu/synergies-disentanglement-sparsity","repo_kind":"official","path":"metaoptnet/models/classification_heads.py","file_url":"https://github.com/tristandeleu/synergies-disentanglement-sparsity/blob/HEAD/metaoptnet/models/classification_heads.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7d8d074dc5077026"}},{"code_sha256_prefix":"a4a27f8e89e92fa7","entry":"block_prox_group_lasso_intercept","repo":"tristandeleu/synergies-disentanglement-sparsity","repo_kind":"official","path":"sparsemeta/utils/sparsity.py","file_url":"https://github.com/tristandeleu/synergies-disentanglement-sparsity/blob/HEAD/sparsemeta/utils/sparsity.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a4a27f8e89e92fa7"}},{"code_sha256_prefix":"fb61241b42c06d73","entry":"count_accuracy","repo":"tristandeleu/synergies-disentanglement-sparsity","repo_kind":"official","path":"metaoptnet/utils.py","file_url":"https://github.com/tristandeleu/synergies-disentanglement-sparsity/blob/HEAD/metaoptnet/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fb61241b42c06d73"}},{"code_sha256_prefix":"55cf3e7015913e49","entry":"find_classes","repo":"tristandeleu/synergies-disentanglement-sparsity","repo_kind":"official","path":"metaoptnet/omniglot_loaders.py","file_url":"https://github.com/tristandeleu/synergies-disentanglement-sparsity/blob/HEAD/metaoptnet/omniglot_loaders.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"55cf3e7015913e49"}},{"code_sha256_prefix":"6c7c0f59f96dc0ff","entry":"get_z_z_hat","repo":"tristandeleu/synergies-disentanglement-sparsity","repo_kind":"official","path":"sparsemeta/disentanglement_metrics.py","file_url":"https://github.com/tristandeleu/synergies-disentanglement-sparsity/blob/HEAD/sparsemeta/disentanglement_metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6c7c0f59f96dc0ff"}},{"code_sha256_prefix":"595ffa0e852c6686","entry":"get_z_z_hat_uniform","repo":"tristandeleu/synergies-disentanglement-sparsity","repo_kind":"official","path":"sparsemeta/disentanglement_metrics.py","file_url":"https://github.com/tristandeleu/synergies-disentanglement-sparsity/blob/HEAD/sparsemeta/disentanglement_metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"595ffa0e852c6686"}},{"code_sha256_prefix":"c10cf2ba82c64b72","entry":"group_support","repo":"tristandeleu/synergies-disentanglement-sparsity","repo_kind":"official","path":"sparsemeta/utils/sparsity.py","file_url":"https://github.com/tristandeleu/synergies-disentanglement-sparsity/blob/HEAD/sparsemeta/utils/sparsity.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c10cf2ba82c64b72"}},{"code_sha256_prefix":"12bd4c927d3ba0d4","entry":"mean_corr_coef_np","repo":"tristandeleu/synergies-disentanglement-sparsity","repo_kind":"official","path":"sparsemeta/disentanglement_metrics.py","file_url":"https://github.com/tristandeleu/synergies-disentanglement-sparsity/blob/HEAD/sparsemeta/disentanglement_metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"12bd4c927d3ba0d4"}},{"code_sha256_prefix":"6db52403976aa70c","entry":"one_hot","repo":"tristandeleu/synergies-disentanglement-sparsity","repo_kind":"official","path":"metaoptnet/models/classification_heads.py","file_url":"https://github.com/tristandeleu/synergies-disentanglement-sparsity/blob/HEAD/metaoptnet/models/classification_heads.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6db52403976aa70c"}},{"code_sha256_prefix":"066897d1a9ead7e7","entry":"prox_group_lasso_intercept","repo":"tristandeleu/synergies-disentanglement-sparsity","repo_kind":"official","path":"sparsemeta/utils/sparsity.py","file_url":"https://github.com/tristandeleu/synergies-disentanglement-sparsity/blob/HEAD/sparsemeta/utils/sparsity.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"066897d1a9ead7e7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}