{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-model-agnostic-dataset-condensation","title":"Towards Model-Agnostic Dataset Condensation by Heterogeneous Models","arxiv_id":"2409.14538","date":"2024-09-22","proceeding":null,"authors":["Jun-Yeong Moon","Jung Uk Kim","Gyeong-Moon Park"],"abstract":"Abstract. The advancement of deep learning has coincided with the proliferation of both models and available data. The surge in dataset sizes and the subsequent surge in computational requirements have led to the development of the Dataset Condensation (DC). While prior studies have delved into generating synthetic images through methods like distribution alignment and training trajectory tracking for more efficient model training, a significant challenge arises when employing these condensed images practically. Notably, these condensed images tend to be specific to particular models, constraining their versatility and practicality. In response to this limitation, we introduce a novel method, Heterogeneous Model Dataset Condensation (HMDC), designed to produce universally applicable condensed images through cross-model interactions. To address the issues of gradient magnitude difference and semantic distance in models when utilizing heterogeneous models, we propose the Gradient Balance Module (GBM) and Mutual Distillation (MD) with the SpatialSemantic Decomposition method. By balancing the contribution of each model and maintaining their semantic meaning closely, our approach overcomes the limitations associated with model-specific condensed images and enhances the broader utility. The source code is available in https://github.com/KHU-AGI/HMDC.","url_abs":"https://arxiv.org/abs/2409.14538v1","url_pdf":"https://arxiv.org/pdf/2409.14538v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-model-agnostic-dataset-condensation","repo_url":"https://github.com/khu-agi/hmdc","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"dataset-condensation","task_name":"Dataset Condensation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2409.14538","atlas_url":"https://app.syntology.ai/?focus=2409.14538","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2409.14538"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/khu-agi/hmdc","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/KHU-AGI/HMDC","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":3,"ran_fixture":3,"unverified":3},"by_repo_kind":{"official":{"samples":9,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"9cb7add8f210b472","entry":"act_layer","repo":"KHU-AGI/HMDC","repo_kind":"official","path":"gcn_lib/torch_nn.py","file_url":"https://github.com/KHU-AGI/HMDC/blob/HEAD/gcn_lib/torch_nn.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9cb7add8f210b472"}},{"code_sha256_prefix":"996eec2334fe011e","entry":"batched_index_select","repo":"KHU-AGI/HMDC","repo_kind":"official","path":"gcn_lib/torch_nn.py","file_url":"https://github.com/KHU-AGI/HMDC/blob/HEAD/gcn_lib/torch_nn.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"996eec2334fe011e"}},{"code_sha256_prefix":"09e6ddb17c0adf28","entry":"norm_layer","repo":"KHU-AGI/HMDC","repo_kind":"official","path":"gcn_lib/torch_nn.py","file_url":"https://github.com/KHU-AGI/HMDC/blob/HEAD/gcn_lib/torch_nn.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"09e6ddb17c0adf28"}},{"code_sha256_prefix":"82aeae314890e7ed","entry":"pairwise_distance","repo":"KHU-AGI/HMDC","repo_kind":"official","path":"gcn_lib/torch_edge.py","file_url":"https://github.com/KHU-AGI/HMDC/blob/HEAD/gcn_lib/torch_edge.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"82aeae314890e7ed"}},{"code_sha256_prefix":"102d0e5b09e9b385","entry":"part_pairwise_distance","repo":"KHU-AGI/HMDC","repo_kind":"official","path":"gcn_lib/torch_edge.py","file_url":"https://github.com/KHU-AGI/HMDC/blob/HEAD/gcn_lib/torch_edge.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"102d0e5b09e9b385"}},{"code_sha256_prefix":"160401504e7d0dbb","entry":"xy_pairwise_distance","repo":"KHU-AGI/HMDC","repo_kind":"official","path":"gcn_lib/torch_edge.py","file_url":"https://github.com/KHU-AGI/HMDC/blob/HEAD/gcn_lib/torch_edge.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"160401504e7d0dbb"}},{"code_sha256_prefix":"571021715428528c","entry":"get_2d_relative_pos_embed","repo":"KHU-AGI/HMDC","repo_kind":"official","path":"gcn_lib/pos_embed.py","file_url":"https://github.com/KHU-AGI/HMDC/blob/HEAD/gcn_lib/pos_embed.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"571021715428528c"}},{"code_sha256_prefix":"3185afc3e87293ed","entry":"get_2d_sincos_pos_embed","repo":"KHU-AGI/HMDC","repo_kind":"official","path":"gcn_lib/pos_embed.py","file_url":"https://github.com/KHU-AGI/HMDC/blob/HEAD/gcn_lib/pos_embed.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3185afc3e87293ed"}},{"code_sha256_prefix":"f10004e059714d42","entry":"get_2d_sincos_pos_embed_from_grid","repo":"KHU-AGI/HMDC","repo_kind":"official","path":"gcn_lib/pos_embed.py","file_url":"https://github.com/KHU-AGI/HMDC/blob/HEAD/gcn_lib/pos_embed.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f10004e059714d42"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}