{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/distill-gold-from-massive-ores-efficient","title":"Distill Gold from Massive Ores: Bi-level Data Pruning towards Efficient Dataset Distillation","arxiv_id":"2305.18381","date":"2023-05-28","proceeding":null,"authors":["Yue Xu","Yong-Lu Li","Kaitong Cui","Ziyu Wang","Cewu Lu","Yu-Wing Tai","Chi-Keung Tang"],"abstract":"Data-efficient learning has garnered significant attention, especially given the current trend of large multi-modal models. Recently, dataset distillation has become an effective approach by synthesizing data samples that are essential for network training. However, it remains to be explored which samples are essential for the dataset distillation process itself. In this work, we study the data efficiency and selection for the dataset distillation task. By re-formulating the dynamics of distillation, we provide insight into the inherent redundancy in the real dataset, both theoretically and empirically. We propose to use the empirical loss value as a static data pruning criterion. To further compensate for the variation of the data value in training, we find the most contributing samples based on their causal effects on the distillation. The proposed selection strategy can efficiently exploit the training dataset, outperform the previous SOTA distillation algorithms, and consistently enhance the distillation algorithms, even on much larger-scale and more heterogeneous datasets, e.g., full ImageNet-1K and Kinetics-400. We believe this paradigm will open up new avenues in the dynamics of distillation and pave the way for efficient dataset distillation. Our code is available on https://github.com/silicx/GoldFromOres-BiLP.","url_abs":"https://arxiv.org/abs/2305.18381v4","url_pdf":"https://arxiv.org/pdf/2305.18381v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"distill-gold-from-massive-ores-efficient","repo_url":"https://github.com/silicx/goldfromores","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"distill-gold-from-massive-ores-efficient","repo_url":"https://github.com/silicx/goldfromores-bilp","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"dataset-distillation","task_name":"Dataset Distillation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2305.18381","atlas_url":"https://app.syntology.ai/?focus=2305.18381","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.18381"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/silicx/goldfromores","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/silicx/goldfromores-bilp","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":1,"unverified":7},"by_repo_kind":{"official":{"samples":8,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6e9c65aaa7c1b5da","entry":"distance_wb","repo":"silicx/goldfromores","repo_kind":"official","path":"DatasetCondensation/utils.py","file_url":"https://github.com/silicx/goldfromores/blob/HEAD/DatasetCondensation/utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6e9c65aaa7c1b5da"}},{"code_sha256_prefix":"8e1b179ef1f0fe03","entry":"VGG11","repo":"silicx/goldfromores","repo_kind":"official","path":"DatasetCondensation/networks.py","file_url":"https://github.com/silicx/goldfromores/blob/HEAD/DatasetCondensation/networks.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8e1b179ef1f0fe03"}},{"code_sha256_prefix":"cfab0fafbf988aa4","entry":"VGG11BN","repo":"silicx/goldfromores","repo_kind":"official","path":"DatasetCondensation/networks.py","file_url":"https://github.com/silicx/goldfromores/blob/HEAD/DatasetCondensation/networks.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cfab0fafbf988aa4"}},{"code_sha256_prefix":"07962bf28fe347a0","entry":"VGG13","repo":"silicx/goldfromores","repo_kind":"official","path":"DatasetCondensation/networks.py","file_url":"https://github.com/silicx/goldfromores/blob/HEAD/DatasetCondensation/networks.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"07962bf28fe347a0"}},{"code_sha256_prefix":"55733ece87768334","entry":"drop_samples","repo":"silicx/goldfromores","repo_kind":"official","path":"drop_utils/drop.py","file_url":"https://github.com/silicx/goldfromores/blob/HEAD/drop_utils/drop.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"55733ece87768334"}},{"code_sha256_prefix":"4ede4e38dc61074f","entry":"get_dataset","repo":"silicx/goldfromores","repo_kind":"official","path":"DatasetCondensation/utils.py","file_url":"https://github.com/silicx/goldfromores/blob/HEAD/DatasetCondensation/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4ede4e38dc61074f"}},{"code_sha256_prefix":"a6dd063f2c93ac08","entry":"get_network","repo":"silicx/goldfromores","repo_kind":"official","path":"DatasetCondensation/utils.py","file_url":"https://github.com/silicx/goldfromores/blob/HEAD/DatasetCondensation/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a6dd063f2c93ac08"}},{"code_sha256_prefix":"4a6b0c194587f5d1","entry":"sample_indices_to_drop","repo":"silicx/goldfromores","repo_kind":"official","path":"drop_utils/drop.py","file_url":"https://github.com/silicx/goldfromores/blob/HEAD/drop_utils/drop.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4a6b0c194587f5d1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}