{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/diffusion-based-generation-optimization-and","title":"Diffusion-based Generation, Optimization, and Planning in 3D Scenes","arxiv_id":"2301.06015","date":"2023-01-15","proceeding":"CVPR 2023 1","authors":["Siyuan Huang","Zan Wang","Puhao Li","Baoxiong Jia","Tengyu Liu","Yixin Zhu","Wei Liang","Song-Chun Zhu"],"abstract":"We introduce SceneDiffuser, a conditional generative model for 3D scene understanding. SceneDiffuser provides a unified model for solving scene-conditioned generation, optimization, and planning. In contrast to prior works, SceneDiffuser is intrinsically scene-aware, physics-based, and goal-oriented. With an iterative sampling strategy, SceneDiffuser jointly formulates the scene-aware generation, physics-based optimization, and goal-oriented planning via a diffusion-based denoising process in a fully differentiable fashion. Such a design alleviates the discrepancies among different modules and the posterior collapse of previous scene-conditioned generative models. We evaluate SceneDiffuser with various 3D scene understanding tasks, including human pose and motion generation, dexterous grasp generation, path planning for 3D navigation, and motion planning for robot arms. The results show significant improvements compared with previous models, demonstrating the tremendous potential of SceneDiffuser for the broad community of 3D scene understanding.","url_abs":"https://arxiv.org/abs/2301.06015v1","url_pdf":"https://arxiv.org/pdf/2301.06015v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"diffusion-based-generation-optimization-and","repo_url":"https://github.com/scenediffuser/Scene-Diffuser","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"diffusion-based-generation-optimization-and","repo_url":"https://github.com/zhouxian/FluidLab","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"grasp-generation","task_name":"Grasp Generation"},{"task_slug":"motion-generation","task_name":"Motion Generation"},{"task_slug":"motion-planning","task_name":"Motion Planning"},{"task_slug":"scene-understanding","task_name":"Scene Understanding"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2301.06015","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2301.06015"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/scenediffuser/Scene-Diffuser","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhouxian/FluidLab","reach":{"status":"ok"}}],"summary":{"ran":4,"ran_fixture":1},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8883fb08d6be0bae","entry":"DDPM","repo":"scenediffuser/Scene-Diffuser","repo_kind":"official","path":"models/dm/ddpm.py","file_url":"https://github.com/scenediffuser/Scene-Diffuser/blob/HEAD/models/dm/ddpm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8883fb08d6be0bae"}},{"code_sha256_prefix":"a6222e5f451c662a","entry":"Optimizer","repo":"scenediffuser/Scene-Diffuser","repo_kind":"official","path":"models/dm/ddpm.py","file_url":"https://github.com/scenediffuser/Scene-Diffuser/blob/HEAD/models/dm/ddpm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a6222e5f451c662a"}},{"code_sha256_prefix":"ba8ebb3d3d0e9755","entry":"Planner","repo":"scenediffuser/Scene-Diffuser","repo_kind":"official","path":"models/dm/ddpm.py","file_url":"https://github.com/scenediffuser/Scene-Diffuser/blob/HEAD/models/dm/ddpm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ba8ebb3d3d0e9755"}},{"code_sha256_prefix":"04c9eb677ef4e5cc","entry":"Registry","repo":"scenediffuser/Scene-Diffuser","repo_kind":"official","path":"models/dm/ddpm.py","file_url":"https://github.com/scenediffuser/Scene-Diffuser/blob/HEAD/models/dm/ddpm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"04c9eb677ef4e5cc"}},{"code_sha256_prefix":"9b2feaa58db0b46e","entry":"make_schedule_ddpm","repo":"scenediffuser/Scene-Diffuser","repo_kind":"official","path":"models/dm/ddpm.py","file_url":"https://github.com/scenediffuser/Scene-Diffuser/blob/HEAD/models/dm/ddpm.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9b2feaa58db0b46e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}