{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/improved-techniques-for-maximum-likelihood","title":"Improved Techniques for Maximum Likelihood Estimation for Diffusion ODEs","arxiv_id":"2305.03935","date":"2023-05-06","proceeding":null,"authors":["Kaiwen Zheng","Cheng Lu","Jianfei Chen","Jun Zhu"],"abstract":"Diffusion models have exhibited excellent performance in various domains. The probability flow ordinary differential equation (ODE) of diffusion models (i.e., diffusion ODEs) is a particular case of continuous normalizing flows (CNFs), which enables deterministic inference and exact likelihood evaluation. However, the likelihood estimation results by diffusion ODEs are still far from those of the state-of-the-art likelihood-based generative models. In this work, we propose several improved techniques for maximum likelihood estimation for diffusion ODEs, including both training and evaluation perspectives. For training, we propose velocity parameterization and explore variance reduction techniques for faster convergence. We also derive an error-bounded high-order flow matching objective for finetuning, which improves the ODE likelihood and smooths its trajectory. For evaluation, we propose a novel training-free truncated-normal dequantization to fill the training-evaluation gap commonly existing in diffusion ODEs. Building upon these techniques, we achieve state-of-the-art likelihood estimation results on image datasets (2.56 on CIFAR-10, 3.43/3.69 on ImageNet-32) without variational dequantization or data augmentation, and 2.42 on CIFAR-10 with data augmentation. Code is available at \\url{https://github.com/thu-ml/i-DODE}.","url_abs":"https://arxiv.org/abs/2305.03935v4","url_pdf":"https://arxiv.org/pdf/2305.03935v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"improved-techniques-for-maximum-likelihood","repo_url":"https://github.com/thu-ml/i-dode","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"density-estimation","task_name":"Density Estimation"},{"task_slug":"image-generation","task_name":"Image Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"normalizing-flows","method_name":"Normalizing Flows"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/density-estimation-on-cifar-10","task":"Density Estimation","dataset":"CIFAR-10","model":"i-DODE","rank_in_archive_order":1,"of":15,"metrics":{"NLL (bits/dim)":"2.42"},"uses_additional_data":false},{"leaderboard":"/sota/density-estimation-on-imagenet-32x32-1","task":"Density Estimation","dataset":"ImageNet 32x32","model":"i-DODE","rank_in_archive_order":2,"of":5,"metrics":{"NLL (bits/dim)":"3.69"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-imagenet-32x32","task":"Image Generation","dataset":"ImageNet 32x32","model":"i-DODE","rank_in_archive_order":9,"of":35,"metrics":{"FID":"9.09","bpd":"3.43"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.03935","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.03935"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thu-ml/i-dode","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_violates":1,"ran":3,"unverified":3},"by_repo_kind":{"official":{"samples":7,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"705aea0955c9050d","entry":"batch_mul","repo":"thu-ml/i-dode","repo_kind":"official","path":"utils.py","file_url":"https://github.com/thu-ml/i-dode/blob/HEAD/utils.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"705aea0955c9050d"}},{"code_sha256_prefix":"d741dec5707d4fea","entry":"copy_dict","repo":"thu-ml/i-dode","repo_kind":"official","path":"experiment.py","file_url":"https://github.com/thu-ml/i-dode/blob/HEAD/experiment.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d741dec5707d4fea"}},{"code_sha256_prefix":"4ff86972ab4194f5","entry":"dist","repo":"thu-ml/i-dode","repo_kind":"official","path":"utils.py","file_url":"https://github.com/thu-ml/i-dode/blob/HEAD/utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4ff86972ab4194f5"}},{"code_sha256_prefix":"428176833658e89d","entry":"restore_partial","repo":"thu-ml/i-dode","repo_kind":"official","path":"experiment.py","file_url":"https://github.com/thu-ml/i-dode/blob/HEAD/experiment.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"428176833658e89d"}},{"code_sha256_prefix":"4c25c1400cdbb0a1","entry":"allgather_and_reshape","repo":"thu-ml/i-dode","repo_kind":"official","path":"utils.py","file_url":"https://github.com/thu-ml/i-dode/blob/HEAD/utils.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4c25c1400cdbb0a1"}},{"code_sha256_prefix":"f25b8c5dd6fba4a7","entry":"dot_product_attention","repo":"thu-ml/i-dode","repo_kind":"official","path":"model_vdm.py","file_url":"https://github.com/thu-ml/i-dode/blob/HEAD/model_vdm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f25b8c5dd6fba4a7"}},{"code_sha256_prefix":"4ca98d7804c64a81","entry":"get_timestep_embedding","repo":"thu-ml/i-dode","repo_kind":"official","path":"model_vdm.py","file_url":"https://github.com/thu-ml/i-dode/blob/HEAD/model_vdm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4ca98d7804c64a81"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}