{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dreaming-to-distill-data-free-knowledge","title":"Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion","arxiv_id":"1912.08795","date":"2019-12-18","proceeding":"CVPR 2020 6","authors":["Hongxu Yin","Pavlo Molchanov","Zhizhong Li","Jose M. Alvarez","Arun Mallya","Derek Hoiem","Niraj K. Jha","Jan Kautz"],"abstract":"We introduce DeepInversion, a new method for synthesizing images from the image distribution used to train a deep neural network. We 'invert' a trained network (teacher) to synthesize class-conditional input images starting from random noise, without using any additional information about the training dataset. Keeping the teacher fixed, our method optimizes the input while regularizing the distribution of intermediate feature maps using information stored in the batch normalization layers of the teacher. Further, we improve the diversity of synthesized images using Adaptive DeepInversion, which maximizes the Jensen-Shannon divergence between the teacher and student network logits. The resulting synthesized images from networks trained on the CIFAR-10 and ImageNet datasets demonstrate high fidelity and degree of realism, and help enable a new breed of data-free applications - ones that do not require any real images or labeled data. We demonstrate the applicability of our proposed method to three tasks of immense practical importance -- (i) data-free network pruning, (ii) data-free knowledge transfer, and (iii) data-free continual learning. Code is available at https://github.com/NVlabs/DeepInversion","url_abs":"https://arxiv.org/abs/1912.08795v2","url_pdf":"https://arxiv.org/pdf/1912.08795v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dreaming-to-distill-data-free-knowledge","repo_url":"https://github.com/NVlabs/DeepInversion","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"dreaming-to-distill-data-free-knowledge","repo_url":"https://github.com/Guy-Shapira/DeepInversion","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"},{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"network-pruning","task_name":"Network Pruning"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[{"method_slug":"batch-normalization","method_name":"Batch Normalization"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1912.08795","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1912.08795"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/NVlabs/DeepInversion","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Guy-Shapira/DeepInversion","reach":{"status":"ok"}}],"summary":{"ran_fixture":1,"unverified":1},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"779603bf93d176de","entry":"get_image_prior_losses","repo":"NVlabs/DeepInversion","repo_kind":"official","path":"deepinversion.py","file_url":"https://github.com/NVlabs/DeepInversion/blob/HEAD/deepinversion.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"779603bf93d176de"}},{"code_sha256_prefix":"5e6851f7d2ba1c3c","entry":"validate_one","repo":"NVlabs/DeepInversion","repo_kind":"official","path":"imagenet_inversion.py","file_url":"https://github.com/NVlabs/DeepInversion/blob/HEAD/imagenet_inversion.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"5e6851f7d2ba1c3c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}