{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/data-free-adversarial-distillation","title":"Data-Free Adversarial Distillation","arxiv_id":"1912.11006","date":"2019-12-23","proceeding":null,"authors":["Gongfan Fang","Jie Song","Chengchao Shen","Xinchao Wang","Da Chen","Mingli Song"],"abstract":"Knowledge Distillation (KD) has made remarkable progress in the last few years and become a popular paradigm for model compression and knowledge transfer. However, almost all existing KD algorithms are data-driven, i.e., relying on a large amount of original training data or alternative data, which is usually unavailable in real-world scenarios. In this paper, we devote ourselves to this challenging problem and propose a novel adversarial distillation mechanism to craft a compact student model without any real-world data. We introduce a model discrepancy to quantificationally measure the difference between student and teacher models and construct an optimizable upper bound. In our work, the student and the teacher jointly act the role of the discriminator to reduce this discrepancy, when a generator adversarially produces some \"hard samples\" to enlarge it. Extensive experiments demonstrate that the proposed data-free method yields comparable performance to existing data-driven methods. More strikingly, our approach can be directly extended to semantic segmentation, which is more complicated than classification, and our approach achieves state-of-the-art results. Code and pretrained models are available at https://github.com/VainF/Data-Free-Adversarial-Distillation.","url_abs":"https://arxiv.org/abs/1912.11006v3","url_pdf":"https://arxiv.org/pdf/1912.11006v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"data-free-adversarial-distillation","repo_url":"https://github.com/VainF/Data-Free-Adversarial-Distillation","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"data-free-adversarial-distillation","repo_url":"https://github.com/temibabs/data-free-adversarial-distillation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"data-free-adversarial-distillation","repo_url":"https://github.com/zju-vipa/KamalEngine","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"model-compression","task_name":"Model Compression"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1912.11006","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1912.11006"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zju-vipa/KamalEngine","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/VainF/Data-Free-Adversarial-Distillation","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/temibabs/data-free-adversarial-distillation","reach":{"status":"ok"}}],"summary":{"ran_honours":1,"unverified":1},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"95d8d4d101aeda06","entry":"test","repo":"VainF/Data-Free-Adversarial-Distillation","repo_kind":"official","path":"train_teacher.py","file_url":"https://github.com/VainF/Data-Free-Adversarial-Distillation/blob/HEAD/train_teacher.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"95d8d4d101aeda06"}},{"code_sha256_prefix":"b113b0620ccf30c3","entry":"test","repo":"VainF/Data-Free-Adversarial-Distillation","repo_kind":"official","path":"DFAD_mnist.py","file_url":"https://github.com/VainF/Data-Free-Adversarial-Distillation/blob/HEAD/DFAD_mnist.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b113b0620ccf30c3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}