{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dataset-distillers-are-good-label-denoisers","title":"Dataset Distillers Are Good Label Denoisers In the Wild","arxiv_id":"2411.11924","date":"2024-11-18","proceeding":null,"authors":["Lechao Cheng","KaiFeng Chen","Jiyang Li","Shengeng Tang","Shufei Zhang","Meng Wang"],"abstract":"Learning from noisy data has become essential for adapting deep learning models to real-world applications. Traditional methods often involve first evaluating the noise and then applying strategies such as discarding noisy samples, re-weighting, or re-labeling. However, these methods can fall into a vicious cycle when the initial noise evaluation is inaccurate, leading to suboptimal performance. To address this, we propose a novel approach that leverages dataset distillation for noise removal. This method avoids the feedback loop common in existing techniques and enhances training efficiency, while also providing strong privacy protection through offline processing. We rigorously evaluate three representative dataset distillation methods (DATM, DANCE, and RCIG) under various noise conditions, including symmetric noise, asymmetric noise, and real-world natural noise. Our empirical findings reveal that dataset distillation effectively serves as a denoising tool in random noise scenarios but may struggle with structured asymmetric noise patterns, which can be absorbed into the distilled samples. Additionally, clean but challenging samples, such as those from tail classes in imbalanced datasets, may undergo lossy compression during distillation. Despite these challenges, our results highlight that dataset distillation holds significant promise for robust model training, especially in high-privacy environments where noise is prevalent. The source code is available at https://github.com/Kciiiman/DD_LNL.","url_abs":"https://arxiv.org/abs/2411.11924v2","url_pdf":"https://arxiv.org/pdf/2411.11924v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dataset-distillers-are-good-label-denoisers","repo_url":"https://github.com/kciiiman/dd_lnl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok"}}],"tasks":[{"task_slug":"dataset-distillation","task_name":"Dataset Distillation"},{"task_slug":"denoising","task_name":"Denoising"}],"methods":[{"method_slug":"dance","method_name":"DANCE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2411.11924","atlas_url":"https://app.syntology.ai/?focus=2411.11924","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.11924"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kciiiman/dd_lnl","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":2,"ran":8,"unverified":9},"by_repo_kind":{"official":{"samples":19,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":19,"samples":[{"code_sha256_prefix":"2813416b75f3a5a8","entry":"remove_aug","repo":"kciiiman/dd_lnl","repo_kind":"official","path":"DANCE+ours/common.py","file_url":"https://github.com/kciiiman/dd_lnl/blob/HEAD/DANCE%2Bours/common.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2813416b75f3a5a8"}},{"code_sha256_prefix":"19442a6a71144dbd","entry":"DenseNet121","repo":"kciiiman/dd_lnl","repo_kind":"official","path":"DANCE+ours/models/densenet_cifar.py","file_url":"https://github.com/kciiiman/dd_lnl/blob/HEAD/DANCE%2Bours/models/densenet_cifar.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"19442a6a71144dbd"}},{"code_sha256_prefix":"96a2132550304d0d","entry":"DenseNet169","repo":"kciiiman/dd_lnl","repo_kind":"official","path":"DANCE+ours/models/densenet_cifar.py","file_url":"https://github.com/kciiiman/dd_lnl/blob/HEAD/DANCE%2Bours/models/densenet_cifar.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"96a2132550304d0d"}},{"code_sha256_prefix":"8937ae0759fe135c","entry":"DenseNet201","repo":"kciiiman/dd_lnl","repo_kind":"official","path":"DANCE+ours/models/densenet_cifar.py","file_url":"https://github.com/kciiiman/dd_lnl/blob/HEAD/DANCE%2Bours/models/densenet_cifar.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8937ae0759fe135c"}},{"code_sha256_prefix":"2e6f03f47dc4131d","entry":"VGG11","repo":"kciiiman/dd_lnl","repo_kind":"official","path":"DATM+ours/networks.py","file_url":"https://github.com/kciiiman/dd_lnl/blob/HEAD/DATM%2Bours/networks.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2e6f03f47dc4131d"}},{"code_sha256_prefix":"cce7c5ae5e902d5b","entry":"VGG11BN","repo":"kciiiman/dd_lnl","repo_kind":"official","path":"DATM+ours/networks.py","file_url":"https://github.com/kciiiman/dd_lnl/blob/HEAD/DATM%2Bours/networks.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cce7c5ae5e902d5b"}},{"code_sha256_prefix":"57fb277537295695","entry":"VGG11_Tiny","repo":"kciiiman/dd_lnl","repo_kind":"official","path":"DATM+ours/networks.py","file_url":"https://github.com/kciiiman/dd_lnl/blob/HEAD/DATM%2Bours/networks.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"57fb277537295695"}},{"code_sha256_prefix":"fac5364e2f53c6db","entry":"conv3x3","repo":"kciiiman/dd_lnl","repo_kind":"official","path":"DANCE+ours/models/resnet.py","file_url":"https://github.com/kciiiman/dd_lnl/blob/HEAD/DANCE%2Bours/models/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fac5364e2f53c6db"}},{"code_sha256_prefix":"8aac35516308515a","entry":"normalization","repo":"kciiiman/dd_lnl","repo_kind":"official","path":"DANCE+ours/models/resnet.py","file_url":"https://github.com/kciiiman/dd_lnl/blob/HEAD/DANCE%2Bours/models/resnet.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8aac35516308515a"}},{"code_sha256_prefix":"52605edc9d326455","entry":"transform_cifar","repo":"kciiiman/dd_lnl","repo_kind":"official","path":"DANCE+ours/data.py","file_url":"https://github.com/kciiiman/dd_lnl/blob/HEAD/DANCE%2Bours/data.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"52605edc9d326455"}},{"code_sha256_prefix":"705d6efdfd6176e7","entry":"VGG11","repo":"kciiiman/dd_lnl","repo_kind":"official","path":"RCIG+ours/models.py","file_url":"https://github.com/kciiiman/dd_lnl/blob/HEAD/RCIG%2Bours/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"705d6efdfd6176e7"}},{"code_sha256_prefix":"69d671dc134c7d4b","entry":"VGG13","repo":"kciiiman/dd_lnl","repo_kind":"official","path":"RCIG+ours/models.py","file_url":"https://github.com/kciiiman/dd_lnl/blob/HEAD/RCIG%2Bours/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"69d671dc134c7d4b"}},{"code_sha256_prefix":"e3f4257b6ec92e4b","entry":"VGG16","repo":"kciiiman/dd_lnl","repo_kind":"official","path":"RCIG+ours/models.py","file_url":"https://github.com/kciiiman/dd_lnl/blob/HEAD/RCIG%2Bours/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e3f4257b6ec92e4b"}},{"code_sha256_prefix":"32bbbf9213d49f02","entry":"get_tree_mask","repo":"kciiiman/dd_lnl","repo_kind":"official","path":"RCIG+ours/algorithms.py","file_url":"https://github.com/kciiiman/dd_lnl/blob/HEAD/RCIG%2Bours/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"32bbbf9213d49f02"}},{"code_sha256_prefix":"c8866f6e0f872a68","entry":"identity","repo":"kciiiman/dd_lnl","repo_kind":"official","path":"RCIG+ours/algorithms.py","file_url":"https://github.com/kciiiman/dd_lnl/blob/HEAD/RCIG%2Bours/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c8866f6e0f872a68"}},{"code_sha256_prefix":"5bd248f46233fcd6","entry":"innerloss","repo":"kciiiman/dd_lnl","repo_kind":"official","path":"DANCE+ours/DANCE.py","file_url":"https://github.com/kciiiman/dd_lnl/blob/HEAD/DANCE%2Bours/DANCE.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5bd248f46233fcd6"}},{"code_sha256_prefix":"1bb9a2b0d91ea4cf","entry":"interloss","repo":"kciiiman/dd_lnl","repo_kind":"official","path":"DANCE+ours/DANCE.py","file_url":"https://github.com/kciiiman/dd_lnl/blob/HEAD/DANCE%2Bours/DANCE.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1bb9a2b0d91ea4cf"}},{"code_sha256_prefix":"5b2e2797a6da2cb5","entry":"transform_mnist","repo":"kciiiman/dd_lnl","repo_kind":"official","path":"DANCE+ours/data.py","file_url":"https://github.com/kciiiman/dd_lnl/blob/HEAD/DANCE%2Bours/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5b2e2797a6da2cb5"}},{"code_sha256_prefix":"4a3eef12c129ad96","entry":"transform_svhn","repo":"kciiiman/dd_lnl","repo_kind":"official","path":"DANCE+ours/data.py","file_url":"https://github.com/kciiiman/dd_lnl/blob/HEAD/DANCE%2Bours/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4a3eef12c129ad96"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}