{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/revisiting-data-free-knowledge-distillation","title":"Revisiting Data-Free Knowledge Distillation with Poisoned Teachers","arxiv_id":"2306.02368","date":"2023-06-04","proceeding":null,"authors":["Junyuan Hong","Yi Zeng","Shuyang Yu","Lingjuan Lyu","Ruoxi Jia","Jiayu Zhou"],"abstract":"Data-free knowledge distillation (KD) helps transfer knowledge from a pre-trained model (known as the teacher model) to a smaller model (known as the student model) without access to the original training data used for training the teacher model. However, the security of the synthetic or out-of-distribution (OOD) data required in data-free KD is largely unknown and under-explored. In this work, we make the first effort to uncover the security risk of data-free KD w.r.t. untrusted pre-trained models. We then propose Anti-Backdoor Data-Free KD (ABD), the first plug-in defensive method for data-free KD methods to mitigate the chance of potential backdoors being transferred. We empirically evaluate the effectiveness of our proposed ABD in diminishing transferred backdoor knowledge while maintaining compatible downstream performances as the vanilla KD. We envision this work as a milestone for alarming and mitigating the potential backdoors in data-free KD. Codes are released at https://github.com/illidanlab/ABD.","url_abs":"https://arxiv.org/abs/2306.02368v1","url_pdf":"https://arxiv.org/pdf/2306.02368v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"revisiting-data-free-knowledge-distillation","repo_url":"https://github.com/illidanlab/abd","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"backdoor-defense-for-data-free-distillation","task_name":"Backdoor Defense for Data-Free Distillation with Poisoned Teachers"},{"task_slug":"data-free-knowledge-distillation","task_name":"Data-free Knowledge Distillation"},{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"}],"methods":[{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.02368","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.02368"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/illidanlab/abd","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":2,"unverified":5},"by_repo_kind":{"official":{"samples":7,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2a80220dabcb742a","entry":"conv1x1","repo":"illidanlab/abd","repo_kind":"official","path":"zskt/models/torch_resnet.py","file_url":"https://github.com/illidanlab/abd/blob/HEAD/zskt/models/torch_resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2a80220dabcb742a"}},{"code_sha256_prefix":"600ff2c45e0de056","entry":"conv3x3","repo":"illidanlab/abd","repo_kind":"official","path":"zskt/models/torch_resnet.py","file_url":"https://github.com/illidanlab/abd/blob/HEAD/zskt/models/torch_resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"600ff2c45e0de056"}},{"code_sha256_prefix":"59c53e8537cfdc53","entry":"comp_accuracy","repo":"illidanlab/abd","repo_kind":"official","path":"cmi/vanilla_kd_ood.py","file_url":"https://github.com/illidanlab/abd/blob/HEAD/cmi/vanilla_kd_ood.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"59c53e8537cfdc53"}},{"code_sha256_prefix":"79e6cfbfb6c1af7e","entry":"evaluate_kd","repo":"illidanlab/abd","repo_kind":"official","path":"zskt/kd_distill.py","file_url":"https://github.com/illidanlab/abd/blob/HEAD/zskt/kd_distill.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"79e6cfbfb6c1af7e"}},{"code_sha256_prefix":"d34a085d71b8f4b5","entry":"rand_bbox","repo":"illidanlab/abd","repo_kind":"official","path":"cmi/vanilla_kd_ood.py","file_url":"https://github.com/illidanlab/abd/blob/HEAD/cmi/vanilla_kd_ood.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d34a085d71b8f4b5"}},{"code_sha256_prefix":"b8b4273b93798ec1","entry":"resnet18","repo":"illidanlab/abd","repo_kind":"official","path":"zskt/models/shufl_resnet.py","file_url":"https://github.com/illidanlab/abd/blob/HEAD/zskt/models/shufl_resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b8b4273b93798ec1"}},{"code_sha256_prefix":"42e550398c376b37","entry":"resnet20","repo":"illidanlab/abd","repo_kind":"official","path":"zskt/models/shufl_resnet.py","file_url":"https://github.com/illidanlab/abd/blob/HEAD/zskt/models/shufl_resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"42e550398c376b37"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}