{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-impact-of-model-zoo-size-and-composition","title":"The Impact of Model Zoo Size and Composition on Weight Space Learning","arxiv_id":"2504.10141","date":"2025-04-14","proceeding":null,"authors":["Damian Falk","Konstantin Schürholt","Damian Borth"],"abstract":"Re-using trained neural network models is a common strategy to reduce training cost and transfer knowledge. Weight space learning - using the weights of trained models as data modality - is a promising new field to re-use populations of pre-trained models for future tasks. Approaches in this field have demonstrated high performance both on model analysis and weight generation tasks. However, until now their learning setup requires homogeneous model zoos where all models share the same exact architecture, limiting their capability to generalize beyond the population of models they saw during training. In this work, we remove this constraint and propose a modification to a common weight space learning method to accommodate training on heterogeneous populations of models. We further investigate the resulting impact of model diversity on generating unseen neural network model weights for zero-shot knowledge transfer. Our extensive experimental evaluation shows that including models with varying underlying image datasets has a high impact on performance and generalization, for both in- and out-of-distribution settings. Code is available on github.com/HSG-AIML/MultiZoo-SANE.","url_abs":"https://arxiv.org/abs/2504.10141v1","url_pdf":"https://arxiv.org/pdf/2504.10141v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-impact-of-model-zoo-size-and-composition","repo_url":"https://github.com/hsg-aiml/multizoo-sane","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"},{"task_slug":"weight-space-learning","task_name":"Weight Space Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2504.10141","atlas_url":"https://app.syntology.ai/?focus=2504.10141","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2504.10141"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/HSG-AIML/MultiZoo-SANE","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hsg-aiml/multizoo-sane","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":6,"unverified":4},"by_repo_kind":{"official":{"samples":10,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"cadc8ba49c6865c0","entry":"clean_checkpoint_keys","repo":"HSG-AIML/MultiZoo-SANE","repo_kind":"official","path":"src/mzsane/datasets/dataset_tokens.py","file_url":"https://github.com/HSG-AIML/MultiZoo-SANE/blob/HEAD/src/mzsane/datasets/dataset_tokens.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cadc8ba49c6865c0"}},{"code_sha256_prefix":"889fe8497b0ba7c8","entry":"permute_model_vector","repo":"HSG-AIML/MultiZoo-SANE","repo_kind":"official","path":"src/mzsane/datasets/augmentations.py","file_url":"https://github.com/HSG-AIML/MultiZoo-SANE/blob/HEAD/src/mzsane/datasets/augmentations.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"889fe8497b0ba7c8"}},{"code_sha256_prefix":"5ac745803cfc2796","entry":"read_properties_from_path","repo":"HSG-AIML/MultiZoo-SANE","repo_kind":"official","path":"src/mzsane/datasets/dataset_base.py","file_url":"https://github.com/HSG-AIML/MultiZoo-SANE/blob/HEAD/src/mzsane/datasets/dataset_base.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5ac745803cfc2796"}},{"code_sha256_prefix":"3b4357d601080776","entry":"read_properties_from_path","repo":"HSG-AIML/MultiZoo-SANE","repo_kind":"official","path":"src/mzsane/datasets/dataset_epochs.py","file_url":"https://github.com/HSG-AIML/MultiZoo-SANE/blob/HEAD/src/mzsane/datasets/dataset_epochs.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3b4357d601080776"}},{"code_sha256_prefix":"ec1ff9919f412754","entry":"test_checkpoint_for_nan","repo":"HSG-AIML/MultiZoo-SANE","repo_kind":"official","path":"src/mzsane/datasets/dataset_auxiliaries.py","file_url":"https://github.com/HSG-AIML/MultiZoo-SANE/blob/HEAD/src/mzsane/datasets/dataset_auxiliaries.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ec1ff9919f412754"}},{"code_sha256_prefix":"d2a9bd8273d8f5d4","entry":"test_checkpoint_with_threshold","repo":"HSG-AIML/MultiZoo-SANE","repo_kind":"official","path":"src/mzsane/datasets/dataset_auxiliaries.py","file_url":"https://github.com/HSG-AIML/MultiZoo-SANE/blob/HEAD/src/mzsane/datasets/dataset_auxiliaries.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d2a9bd8273d8f5d4"}},{"code_sha256_prefix":"31ce1adb9a55256c","entry":"get_net_epoch_lst_from_label","repo":"HSG-AIML/MultiZoo-SANE","repo_kind":"official","path":"src/mzsane/datasets/dataset_auxiliaries.py","file_url":"https://github.com/HSG-AIML/MultiZoo-SANE/blob/HEAD/src/mzsane/datasets/dataset_auxiliaries.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"31ce1adb9a55256c"}},{"code_sha256_prefix":"336c00ba04cc94cf","entry":"load_checkpoint","repo":"HSG-AIML/MultiZoo-SANE","repo_kind":"official","path":"src/mzsane/datasets/dataset_base.py","file_url":"https://github.com/HSG-AIML/MultiZoo-SANE/blob/HEAD/src/mzsane/datasets/dataset_base.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"336c00ba04cc94cf"}},{"code_sha256_prefix":"5fb96d161e21926b","entry":"load_checkpoints_remote","repo":"HSG-AIML/MultiZoo-SANE","repo_kind":"official","path":"src/mzsane/datasets/dataset_base.py","file_url":"https://github.com/HSG-AIML/MultiZoo-SANE/blob/HEAD/src/mzsane/datasets/dataset_base.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5fb96d161e21926b"}},{"code_sha256_prefix":"790ee3cda94b46c4","entry":"read_properties_from_path","repo":"HSG-AIML/MultiZoo-SANE","repo_kind":"official","path":"src/mzsane/datasets/dataset_properties.py","file_url":"https://github.com/HSG-AIML/MultiZoo-SANE/blob/HEAD/src/mzsane/datasets/dataset_properties.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"790ee3cda94b46c4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}