{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/task-groupings-regularization-data-free-meta","title":"Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Models","arxiv_id":"2405.16560","date":"2024-05-26","proceeding":null,"authors":["Yongxian Wei","Zixuan Hu","Li Shen","Zhenyi Wang","Yu Li","Chun Yuan","DaCheng Tao"],"abstract":"Data-Free Meta-Learning (DFML) aims to derive knowledge from a collection of pre-trained models without accessing their original data, enabling the rapid adaptation to new unseen tasks. Current methods often overlook the heterogeneity among pre-trained models, which leads to performance degradation due to task conflicts. In this paper, we empirically and theoretically identify and analyze the model heterogeneity in DFML. We find that model heterogeneity introduces a heterogeneity-homogeneity trade-off, where homogeneous models reduce task conflicts but also increase the overfitting risk. Balancing this trade-off is crucial for learning shared representations across tasks. Based on our findings, we propose Task Groupings Regularization that benefits from model heterogeneity by grouping and aligning conflicting tasks. Specifically, we embed pre-trained models into a task space to compute dissimilarity, and group heterogeneous models together based on this measure. Then, we introduce implicit gradient regularization within each group to mitigate potential conflicts. By encouraging a gradient direction suitable for all tasks, the meta-model captures shared representations that generalize across tasks. Comprehensive experiments showcase the superiority of our approach in multiple benchmarks, effectively tackling the model heterogeneity in challenging multi-domain and multi-architecture scenarios.","url_abs":"https://arxiv.org/abs/2405.16560v2","url_pdf":"https://arxiv.org/pdf/2405.16560v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"task-groupings-regularization-data-free-meta","repo_url":"https://github.com/walkerworldpeace/tgr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"meta-learning","task_name":"Meta-Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2405.16560","atlas_url":"https://app.syntology.ai/?focus=2405.16560","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.16560"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/walkerworldpeace/tgr","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":4,"ran_honours":1,"ran_fixture":1,"unverified":3},"by_repo_kind":{"official":{"samples":9,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5e18ddf34031af1d","entry":"calculate_overlap","repo":"walkerworldpeace/tgr","repo_kind":"official","path":"empirical.py","file_url":"https://github.com/walkerworldpeace/tgr/blob/HEAD/empirical.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5e18ddf34031af1d"}},{"code_sha256_prefix":"4db0088e96af2c78","entry":"custom_collate_fn","repo":"walkerworldpeace/tgr","repo_kind":"official","path":"empirical.py","file_url":"https://github.com/walkerworldpeace/tgr/blob/HEAD/empirical.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4db0088e96af2c78"}},{"code_sha256_prefix":"109c8b212a5c9895","entry":"fisher","repo":"walkerworldpeace/tgr","repo_kind":"official","path":"task_similarity.py","file_url":"https://github.com/walkerworldpeace/tgr/blob/HEAD/task_similarity.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"109c8b212a5c9895"}},{"code_sha256_prefix":"9910e2fc297f8665","entry":"identity","repo":"walkerworldpeace/tgr","repo_kind":"official","path":"dataset/miniimagenet.py","file_url":"https://github.com/walkerworldpeace/tgr/blob/HEAD/dataset/miniimagenet.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9910e2fc297f8665"}},{"code_sha256_prefix":"b8c9308d810cac84","entry":"normalize","repo":"walkerworldpeace/tgr","repo_kind":"official","path":"synthesis/_utils.py","file_url":"https://github.com/walkerworldpeace/tgr/blob/HEAD/synthesis/_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b8c9308d810cac84"}},{"code_sha256_prefix":"47a55458e0090d06","entry":"pack_images","repo":"walkerworldpeace/tgr","repo_kind":"official","path":"synthesis/_utils.py","file_url":"https://github.com/walkerworldpeace/tgr/blob/HEAD/synthesis/_utils.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"47a55458e0090d06"}},{"code_sha256_prefix":"49fdd0ae186afdb8","entry":"Conv4","repo":"walkerworldpeace/tgr","repo_kind":"official","path":"network.py","file_url":"https://github.com/walkerworldpeace/tgr/blob/HEAD/network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"49fdd0ae186afdb8"}},{"code_sha256_prefix":"1c7f99c71103253a","entry":"ResNet10","repo":"walkerworldpeace/tgr","repo_kind":"official","path":"network.py","file_url":"https://github.com/walkerworldpeace/tgr/blob/HEAD/network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1c7f99c71103253a"}},{"code_sha256_prefix":"8a52b4ddf8c6b7c2","entry":"ResNet18","repo":"walkerworldpeace/tgr","repo_kind":"official","path":"network.py","file_url":"https://github.com/walkerworldpeace/tgr/blob/HEAD/network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8a52b4ddf8c6b7c2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}