{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2512-15647","title":"Hard Labels In! Rethinking the Role of Hard Labels in Mitigating Local Semantic Drift","arxiv_id":"2512.15647","date":"2025-12-17","proceeding":"ICML","authors":["Jiacheng Cui","Bingkui Tong","Xinyue Bi","Xiaohan Zhao","Jiacheng Liu","Zhiqiang Shen"],"abstract":"Soft labels from teacher models are a de facto practice for knowledge transfer and large-scale dataset distillation (e.g., SRe2L, LPLD). However, when we limit the number of crops per image to reduce the substantial cost of storing precomputed soft labels, these methods suffer severely from local semantic drift: visually ambiguous crops can cause soft supervision to deviate from the image-level ground-truth semantics, leading to persistent errors and a train-test distribution mismatch. We revisit the overlooked role of hard labels and show that, when properly integrated, they can act as a content-invariant semantic anchor that calibrates such drift. We theoretically analyze the emergence of drift under sparse soft-label supervision and demonstrate that hybridizing hard and soft labels restores alignment between visual content and semantic supervision. Building on this insight, we propose a new training paradigm, Hard Label for Alleviating Local Semantic Drift (HALD), which uses hard labels as intermediate corrective signals while preserving the fine-grained benefits of soft labels. Extensive experiments on dataset distillation and large-scale classification benchmarks show consistent generalization improvements. On ImageNet-1K, our method achieves 42.7% accuracy with only 285M soft-label storage (reduces by 100X), outperforming prior state-of-the-art LPLD 9.0%.","url_abs":"https://arxiv.org/abs/2512.15647","url_pdf":"https://arxiv.org/pdf/2512.15647","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2512.15647","atlas_url":"https://app.syntology.ai/?focus=2512.15647","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2512.15647"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/Jiacheng8/HALD","reach":null}],"summary":{"ran_fixture":1,"ran_violates":1,"ran_honours":1,"unverified":5},"by_repo_kind":{"found_in_text":{"samples":8,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f8bfba92d8328d35","entry":"apply_fkd_transform_batch_tensor","repo":"Jiacheng8/HALD","repo_kind":"found_in_text","path":"validate/train_fkd.py","file_url":"https://github.com/Jiacheng8/HALD/blob/HEAD/validate/train_fkd.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f8bfba92d8328d35"}},{"code_sha256_prefix":"cd4d8e43e2ff286b","entry":"is_special_epoch","repo":"Jiacheng8/HALD","repo_kind":"found_in_text","path":"validate/train_fkd.py","file_url":"https://github.com/Jiacheng8/HALD/blob/HEAD/validate/train_fkd.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cd4d8e43e2ff286b"}},{"code_sha256_prefix":"cef161c1e3365a6a","entry":"soft_cross_entropy","repo":"Jiacheng8/HALD","repo_kind":"found_in_text","path":"validate/train_fkd.py","file_url":"https://github.com/Jiacheng8/HALD/blob/HEAD/validate/train_fkd.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cef161c1e3365a6a"}},{"code_sha256_prefix":"fc95eb37eccb57be","entry":"hard_train","repo":"Jiacheng8/HALD","repo_kind":"found_in_text","path":"validate/train_fkd.py","file_url":"https://github.com/Jiacheng8/HALD/blob/HEAD/validate/train_fkd.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fc95eb37eccb57be"}},{"code_sha256_prefix":"0ba64fb9bbc21cb0","entry":"run_one_epoch","repo":"Jiacheng8/HALD","repo_kind":"found_in_text","path":"validate/train_fkd.py","file_url":"https://github.com/Jiacheng8/HALD/blob/HEAD/validate/train_fkd.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0ba64fb9bbc21cb0"}},{"code_sha256_prefix":"9a03064558c4c3e2","entry":"save_checkpoint","repo":"Jiacheng8/HALD","repo_kind":"found_in_text","path":"validate/train_fkd.py","file_url":"https://github.com/Jiacheng8/HALD/blob/HEAD/validate/train_fkd.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a03064558c4c3e2"}},{"code_sha256_prefix":"4b437fdad717871d","entry":"soft_train","repo":"Jiacheng8/HALD","repo_kind":"found_in_text","path":"validate/train_fkd.py","file_url":"https://github.com/Jiacheng8/HALD/blob/HEAD/validate/train_fkd.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4b437fdad717871d"}},{"code_sha256_prefix":"4d2b9fa1e962b322","entry":"validate","repo":"Jiacheng8/HALD","repo_kind":"found_in_text","path":"validate/train_fkd.py","file_url":"https://github.com/Jiacheng8/HALD/blob/HEAD/validate/train_fkd.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4d2b9fa1e962b322"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CV","source":"arxiv_api"},"syntology_extracted_results":null}