{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/exploiting-inter-sample-and-inter-feature","title":"Exploiting Inter-sample and Inter-feature Relations in Dataset Distillation","arxiv_id":"2404.00563","date":"2024-03-31","proceeding":"CVPR 2024 1","authors":["Wenxiao Deng","Wenbin Li","Tianyu Ding","Lei Wang","Hongguang Zhang","Kuihua Huang","Jing Huo","Yang Gao"],"abstract":"Dataset distillation has emerged as a promising approach in deep learning, enabling efficient training with small synthetic datasets derived from larger real ones. Particularly, distribution matching-based distillation methods attract attention thanks to its effectiveness and low computational cost. However, these methods face two primary limitations: the dispersed feature distribution within the same class in synthetic datasets, reducing class discrimination, and an exclusive focus on mean feature consistency, lacking precision and comprehensiveness. To address these challenges, we introduce two novel constraints: a class centralization constraint and a covariance matching constraint. The class centralization constraint aims to enhance class discrimination by more closely clustering samples within classes. The covariance matching constraint seeks to achieve more accurate feature distribution matching between real and synthetic datasets through local feature covariance matrices, particularly beneficial when sample sizes are much smaller than the number of features. Experiments demonstrate notable improvements with these constraints, yielding performance boosts of up to 6.6% on CIFAR10, 2.9% on SVHN, 2.5% on CIFAR100, and 2.5% on TinyImageNet, compared to the state-of-the-art relevant methods. In addition, our method maintains robust performance in cross-architecture settings, with a maximum performance drop of 1.7% on four architectures. Code is available at https://github.com/VincenDen/IID.","url_abs":"https://arxiv.org/abs/2404.00563v1","url_pdf":"https://arxiv.org/pdf/2404.00563v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"exploiting-inter-sample-and-inter-feature","repo_url":"https://github.com/vincenden/iid","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"dataset-distillation","task_name":"Dataset Distillation"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2404.00563","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.00563"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vincenden/iid","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/VincenDen/IID","reach":{"status":"ok"}}],"summary":{"ran":6,"ran_honours":1,"unverified":5},"by_repo_kind":{"official":{"samples":12,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":12,"samples":[{"code_sha256_prefix":"190dfb67aacf4767","entry":"VGG11","repo":"VincenDen/IID","repo_kind":"official","path":"IDM+ours/networks.py","file_url":"https://github.com/VincenDen/IID/blob/HEAD/IDM%2Bours/networks.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"190dfb67aacf4767"}},{"code_sha256_prefix":"ec3c9224d4a0353f","entry":"VGG11","repo":"VincenDen/IID","repo_kind":"official","path":"IDM+ours/dc_networks.py","file_url":"https://github.com/VincenDen/IID/blob/HEAD/IDM%2Bours/dc_networks.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ec3c9224d4a0353f"}},{"code_sha256_prefix":"6fa9b9b022f6dbfa","entry":"VGG11BN","repo":"vincenden/iid","repo_kind":"official","path":"IDM+ours/networks.py","file_url":"https://github.com/vincenden/iid/blob/HEAD/IDM%2Bours/networks.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6fa9b9b022f6dbfa"}},{"code_sha256_prefix":"01d3ad663f463d41","entry":"VGG11BN","repo":"VincenDen/IID","repo_kind":"official","path":"IDM+ours/dc_networks.py","file_url":"https://github.com/VincenDen/IID/blob/HEAD/IDM%2Bours/dc_networks.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"01d3ad663f463d41"}},{"code_sha256_prefix":"3da7c8c828c4f0af","entry":"VGG13","repo":"vincenden/iid","repo_kind":"official","path":"IDM+ours/networks.py","file_url":"https://github.com/vincenden/iid/blob/HEAD/IDM%2Bours/networks.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3da7c8c828c4f0af"}},{"code_sha256_prefix":"a21845ac7e36313f","entry":"VGG13","repo":"VincenDen/IID","repo_kind":"official","path":"IDM+ours/dc_networks.py","file_url":"https://github.com/VincenDen/IID/blob/HEAD/IDM%2Bours/dc_networks.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a21845ac7e36313f"}},{"code_sha256_prefix":"6e9c65aaa7c1b5da","entry":"distance_wb","repo":"VincenDen/IID","repo_kind":"official","path":"DM+ours/utils.py","file_url":"https://github.com/VincenDen/IID/blob/HEAD/DM%2Bours/utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6e9c65aaa7c1b5da"}},{"code_sha256_prefix":"8e1b179ef1f0fe03","entry":"VGG11","repo":"VincenDen/IID","repo_kind":"official","path":"DM+ours/networks.py","file_url":"https://github.com/VincenDen/IID/blob/HEAD/DM%2Bours/networks.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8e1b179ef1f0fe03"}},{"code_sha256_prefix":"cfab0fafbf988aa4","entry":"VGG11BN","repo":"VincenDen/IID","repo_kind":"official","path":"DM+ours/networks.py","file_url":"https://github.com/VincenDen/IID/blob/HEAD/DM%2Bours/networks.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cfab0fafbf988aa4"}},{"code_sha256_prefix":"07962bf28fe347a0","entry":"VGG13","repo":"VincenDen/IID","repo_kind":"official","path":"DM+ours/networks.py","file_url":"https://github.com/VincenDen/IID/blob/HEAD/DM%2Bours/networks.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"07962bf28fe347a0"}},{"code_sha256_prefix":"4ede4e38dc61074f","entry":"get_dataset","repo":"VincenDen/IID","repo_kind":"official","path":"DM+ours/utils.py","file_url":"https://github.com/VincenDen/IID/blob/HEAD/DM%2Bours/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4ede4e38dc61074f"}},{"code_sha256_prefix":"3b08d76e23b09de8","entry":"get_dataset","repo":"VincenDen/IID","repo_kind":"official","path":"IDM+ours/dc_utils.py","file_url":"https://github.com/VincenDen/IID/blob/HEAD/IDM%2Bours/dc_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3b08d76e23b09de8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}