{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/going-beyond-feature-similarity-effective","title":"Going Beyond Feature Similarity: Effective Dataset Distillation based on Class-Aware Conditional Mutual Information","arxiv_id":"2412.09945","date":"2024-12-13","proceeding":null,"authors":["Xinhao Zhong","Bin Chen","Hao Fang","Xulin Gu","Shu-Tao Xia","En-hui Yang"],"abstract":"Dataset distillation (DD) aims to minimize the time and memory consumption needed for training deep neural networks on large datasets, by creating a smaller synthetic dataset that has similar performance to that of the full real dataset. However, current dataset distillation methods often result in synthetic datasets that are excessively difficult for networks to learn from, due to the compression of a substantial amount of information from the original data through metrics measuring feature similarity, e,g., distribution matching (DM). In this work, we introduce conditional mutual information (CMI) to assess the class-aware complexity of a dataset and propose a novel method by minimizing CMI. Specifically, we minimize the distillation loss while constraining the class-aware complexity of the synthetic dataset by minimizing its empirical CMI from the feature space of pre-trained networks, simultaneously. Conducting on a thorough set of experiments, we show that our method can serve as a general regularization method to existing DD methods and improve the performance and training efficiency.","url_abs":"https://arxiv.org/abs/2412.09945v4","url_pdf":"https://arxiv.org/pdf/2412.09945v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"going-beyond-feature-similarity-effective","repo_url":"https://github.com/ndhg1213/cmidd","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"dataset-distillation","task_name":"Dataset Distillation"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2412.09945","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.09945"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/ndhg1213/CMIDD","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ndhg1213/cmidd","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":1,"ran_honours":1,"unverified":7},"by_repo_kind":{"official":{"samples":8,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"2813416b75f3a5a8","entry":"remove_aug","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"2813416b75f3a5a8"}},{"code_sha256_prefix":"6e9c65aaa7c1b5da","entry":"distance_wb","repo":"ndhg1213/CMIDD","repo_kind":"official","path":"DC-DSA-DM/utils.py","file_url":"https://github.com/ndhg1213/CMIDD/blob/HEAD/DC-DSA-DM/utils.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6e9c65aaa7c1b5da"}},{"code_sha256_prefix":"8e1b179ef1f0fe03","entry":"VGG11","repo":"ndhg1213/CMIDD","repo_kind":"official","path":"DC-DSA-DM/networks.py","file_url":"https://github.com/ndhg1213/CMIDD/blob/HEAD/DC-DSA-DM/networks.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8e1b179ef1f0fe03"}},{"code_sha256_prefix":"cfab0fafbf988aa4","entry":"VGG11BN","repo":"ndhg1213/CMIDD","repo_kind":"official","path":"DC-DSA-DM/networks.py","file_url":"https://github.com/ndhg1213/CMIDD/blob/HEAD/DC-DSA-DM/networks.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cfab0fafbf988aa4"}},{"code_sha256_prefix":"07962bf28fe347a0","entry":"VGG13","repo":"ndhg1213/CMIDD","repo_kind":"official","path":"DC-DSA-DM/networks.py","file_url":"https://github.com/ndhg1213/CMIDD/blob/HEAD/DC-DSA-DM/networks.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"07962bf28fe347a0"}},{"code_sha256_prefix":"2bccb1da81dcb4b1","entry":"get_dataset","repo":"ndhg1213/CMIDD","repo_kind":"official","path":"DC-DSA-DM/utils.py","file_url":"https://github.com/ndhg1213/CMIDD/blob/HEAD/DC-DSA-DM/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2bccb1da81dcb4b1"}},{"code_sha256_prefix":"27c45694acd469da","entry":"get_features","repo":"ndhg1213/CMIDD","repo_kind":"official","path":"IDC/coreset.py","file_url":"https://github.com/ndhg1213/CMIDD/blob/HEAD/IDC/coreset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"27c45694acd469da"}},{"code_sha256_prefix":"01d05ec4434a0ed3","entry":"get_premodel","repo":"ndhg1213/CMIDD","repo_kind":"official","path":"IDC/CMI_utils.py","file_url":"https://github.com/ndhg1213/CMIDD/blob/HEAD/IDC/CMI_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"01d05ec4434a0ed3"}},{"code_sha256_prefix":"e9adb33e803602e6","entry":"remove_prefix_checkpoint","repo":"ndhg1213/CMIDD","repo_kind":"official","path":"IDC/coreset.py","file_url":"https://github.com/ndhg1213/CMIDD/blob/HEAD/IDC/coreset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e9adb33e803602e6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}