{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/one-diffusion-to-generate-them-all","title":"One Diffusion to Generate Them All","arxiv_id":"2411.16318","date":"2024-11-25","proceeding":"CVPR 2025 1","authors":["Duong H. Le","Tuan Pham","Sangho Lee","Christopher Clark","Aniruddha Kembhavi","Stephan Mandt","Ranjay Krishna","Jiasen Lu"],"abstract":"We introduce OneDiffusion, a versatile, large-scale diffusion model that seamlessly supports bidirectional image synthesis and understanding across diverse tasks. It enables conditional generation from inputs such as text, depth, pose, layout, and semantic maps, while also handling tasks like image deblurring, upscaling, and reverse processes such as depth estimation and segmentation. Additionally, OneDiffusion allows for multi-view generation, camera pose estimation, and instant personalization using sequential image inputs. Our model takes a straightforward yet effective approach by treating all tasks as frame sequences with varying noise scales during training, allowing any frame to act as a conditioning image at inference time. Our unified training framework removes the need for specialized architectures, supports scalable multi-task training, and adapts smoothly to any resolution, enhancing both generalization and scalability. Experimental results demonstrate competitive performance across tasks in both generation and prediction such as text-to-image, multiview generation, ID preservation, depth estimation and camera pose estimation despite relatively small training dataset. Our code and checkpoint are freely available at https://github.com/lehduong/OneDiffusion","url_abs":"https://arxiv.org/abs/2411.16318v1","url_pdf":"https://arxiv.org/pdf/2411.16318v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"one-diffusion-to-generate-them-all","repo_url":"https://github.com/lehduong/onediffusion","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"all","task_name":"All"},{"task_slug":"camera-pose-estimation","task_name":"Camera Pose Estimation"},{"task_slug":"deblurring","task_name":"Deblurring"},{"task_slug":"depth-estimation","task_name":"Depth Estimation"},{"task_slug":"image-deblurring","task_name":"Image Deblurring"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"pose-estimation","task_name":"Pose Estimation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2411.16318","atlas_url":"https://app.syntology.ai/?focus=2411.16318","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.16318"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lehduong/onediffusion","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran_violates":1,"ran_draft_wrong":2,"ran_fixture":1,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"4e3ad3679e5c37b2","entry":"calculate_shift","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"4e3ad3679e5c37b2"}},{"code_sha256_prefix":"232db38056e4f799","entry":"colorize_depth_maps","repo":"lehduong/onediffusion","repo_kind":"official","path":"gradio_demo.py","file_url":"https://github.com/lehduong/onediffusion/blob/HEAD/gradio_demo.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"232db38056e4f799"}},{"code_sha256_prefix":"5b46109fe6ae37f0","entry":"create_c2w_matrix","repo":"lehduong/onediffusion","repo_kind":"official","path":"onediffusion/diffusion/pipelines/onediffusion.py","file_url":"https://github.com/lehduong/onediffusion/blob/HEAD/onediffusion/diffusion/pipelines/onediffusion.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"5b46109fe6ae37f0"}},{"code_sha256_prefix":"362b432acdaa105e","entry":"format_prompt","repo":"lehduong/onediffusion","repo_kind":"official","path":"gradio_demo.py","file_url":"https://github.com/lehduong/onediffusion/blob/HEAD/gradio_demo.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"362b432acdaa105e"}},{"code_sha256_prefix":"face06fc05f7e12b","entry":"retrieve_latents","repo":"lehduong/onediffusion","repo_kind":"official","path":"onediffusion/diffusion/pipelines/onediffusion.py","file_url":"https://github.com/lehduong/onediffusion/blob/HEAD/onediffusion/diffusion/pipelines/onediffusion.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"face06fc05f7e12b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}