{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/physics-informed-distillation-for-diffusion","title":"Physics Informed Distillation for Diffusion Models","arxiv_id":"2411.08378","date":"2024-11-13","proceeding":null,"authors":["Joshua Tian Jin Tee","Kang Zhang","Hee Suk Yoon","Dhananjaya Nagaraja Gowda","Chanwoo Kim","Chang D. Yoo"],"abstract":"Diffusion models have recently emerged as a potent tool in generative modeling. However, their inherent iterative nature often results in sluggish image generation due to the requirement for multiple model evaluations. Recent progress has unveiled the intrinsic link between diffusion models and Probability Flow Ordinary Differential Equations (ODEs), thus enabling us to conceptualize diffusion models as ODE systems. Simultaneously, Physics Informed Neural Networks (PINNs) have substantiated their effectiveness in solving intricate differential equations through implicit modeling of their solutions. Building upon these foundational insights, we introduce Physics Informed Distillation (PID), which employs a student model to represent the solution of the ODE system corresponding to the teacher diffusion model, akin to the principles employed in PINNs. Through experiments on CIFAR 10 and ImageNet 64x64, we observe that PID achieves performance comparable to recent distillation methods. Notably, it demonstrates predictable trends concerning method-specific hyperparameters and eliminates the need for synthetic dataset generation during the distillation process. Both of which contribute to its easy-to-use nature as a distillation approach for Diffusion Models. Our code and pre-trained checkpoint are publicly available at: https://github.com/pantheon5100/pid_diffusion.git.","url_abs":"https://arxiv.org/abs/2411.08378v1","url_pdf":"https://arxiv.org/pdf/2411.08378v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"physics-informed-distillation-for-diffusion","repo_url":"https://github.com/pantheon5100/pid_diffusion","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"dataset-generation","task_name":"Dataset Generation"},{"task_slug":"image-generation","task_name":"Image Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2411.08378","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.08378"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pantheon5100/pid_diffusion","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":4,"ran":4,"ran_honours":1,"ran_fixture":1,"unverified":14},"by_repo_kind":{"official":{"samples":24,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8ab9a97df8161f31","entry":"center_crop_arr","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/image_datasets.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/image_datasets.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8ab9a97df8161f31"}},{"code_sha256_prefix":"cfd76fd0d89574a4","entry":"approx_standard_normal_cdf","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/losses.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/losses.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cfd76fd0d89574a4"}},{"code_sha256_prefix":"cd33283d615fb3d7","entry":"discretized_gaussian_log_likelihood","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/losses.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/losses.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cd33283d615fb3d7"}},{"code_sha256_prefix":"e41367ad14ff58fd","entry":"get_param_groups_and_shapes","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/fp16_util.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/fp16_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e41367ad14ff58fd"}},{"code_sha256_prefix":"e20dd5102da3b050","entry":"make_master_params","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/fp16_util.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/fp16_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e20dd5102da3b050"}},{"code_sha256_prefix":"bcd8b4acab199405","entry":"make_output_format","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/logger.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/logger.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bcd8b4acab199405"}},{"code_sha256_prefix":"cf2798b666b231ca","entry":"normal_kl","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/losses.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/losses.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cf2798b666b231ca"}},{"code_sha256_prefix":"64fff1e30802b815","entry":"unflatten_master_params","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/fp16_util.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/fp16_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"64fff1e30802b815"}},{"code_sha256_prefix":"d41a4250066bce93","entry":"weight_init","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/network.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/network.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d41a4250066bce93"}},{"code_sha256_prefix":"129b804760b3115f","entry":"zero_module","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/nn.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/nn.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"129b804760b3115f"}},{"code_sha256_prefix":"ecd0fc28815b65ae","entry":"avg_pool_nd","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/nn.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/nn.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ecd0fc28815b65ae"}},{"code_sha256_prefix":"fe4eb545bbb728e0","entry":"conv_nd","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/nn.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/nn.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fe4eb545bbb728e0"}},{"code_sha256_prefix":"35ea054532112e53","entry":"create_model","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/script_util.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/script_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"35ea054532112e53"}},{"code_sha256_prefix":"a320073abc54d8b8","entry":"create_model_and_diffusion","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/script_util.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/script_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a320073abc54d8b8"}},{"code_sha256_prefix":"df72d0c85cc7f5ed","entry":"create_named_schedule_sampler","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/resample.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/resample.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"df72d0c85cc7f5ed"}},{"code_sha256_prefix":"1514fb2caf08b7f4","entry":"create_one_shot_edmedm_model_and_diffusion","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/script_util.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/script_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1514fb2caf08b7f4"}},{"code_sha256_prefix":"84423ecf52775c17","entry":"get_generator","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/random_util.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/random_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"84423ecf52775c17"}},{"code_sha256_prefix":"b50e02eda098b1ea","entry":"get_weightings","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/karras_diffusion.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/karras_diffusion.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b50e02eda098b1ea"}},{"code_sha256_prefix":"e8556cad8bd4d94a","entry":"karras_sample","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/karras_diffusion.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/karras_diffusion.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e8556cad8bd4d94a"}},{"code_sha256_prefix":"e515a67f7f32e76d","entry":"mpi_weighted_mean","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/logger.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/logger.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e515a67f7f32e76d"}},{"code_sha256_prefix":"2cf07b1cef30b589","entry":"ode_solver","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/karras_diffusion.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/karras_diffusion.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2cf07b1cef30b589"}},{"code_sha256_prefix":"d5307bbc83ea6a98","entry":"our_softmax","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/network.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d5307bbc83ea6a98"}},{"code_sha256_prefix":"0ee9606f5fea937d","entry":"profile","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/logger.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/logger.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0ee9606f5fea937d"}},{"code_sha256_prefix":"53d5be4fdbcfac23","entry":"random_crop_arr","repo":"pantheon5100/pid_diffusion","repo_kind":"official","path":"cm/image_datasets.py","file_url":"https://github.com/pantheon5100/pid_diffusion/blob/HEAD/cm/image_datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"53d5be4fdbcfac23"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}