{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/scenecraft-layout-guided-3d-scene-generation","title":"SceneCraft: Layout-Guided 3D Scene Generation","arxiv_id":"2410.09049","date":"2024-10-11","proceeding":null,"authors":["Xiuyu Yang","Yunze Man","Jun-Kun Chen","Yu-Xiong Wang"],"abstract":"The creation of complex 3D scenes tailored to user specifications has been a tedious and challenging task with traditional 3D modeling tools. Although some pioneering methods have achieved automatic text-to-3D generation, they are generally limited to small-scale scenes with restricted control over the shape and texture. We introduce SceneCraft, a novel method for generating detailed indoor scenes that adhere to textual descriptions and spatial layout preferences provided by users. Central to our method is a rendering-based technique, which converts 3D semantic layouts into multi-view 2D proxy maps. Furthermore, we design a semantic and depth conditioned diffusion model to generate multi-view images, which are used to learn a neural radiance field (NeRF) as the final scene representation. Without the constraints of panorama image generation, we surpass previous methods in supporting complicated indoor space generation beyond a single room, even as complicated as a whole multi-bedroom apartment with irregular shapes and layouts. Through experimental analysis, we demonstrate that our method significantly outperforms existing approaches in complex indoor scene generation with diverse textures, consistent geometry, and realistic visual quality. Code and more results are available at: https://orangesodahub.github.io/SceneCraft","url_abs":"https://arxiv.org/abs/2410.09049v3","url_pdf":"https://arxiv.org/pdf/2410.09049v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"scenecraft-layout-guided-3d-scene-generation","repo_url":"https://github.com/orangesodahub/scenecraft","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"3d-generation","task_name":"3D Generation"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"nerf","task_name":"NeRF"},{"task_slug":"scene-generation","task_name":"Scene Generation"},{"task_slug":"text-to-3d","task_name":"Text to 3D"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.09049","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.09049"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/orangesodahub/scenecraft","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":5,"ran_honours":1,"ran_fixture":1,"unverified":1},"by_repo_kind":{"official":{"samples":8,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"464e03da63a630e1","entry":"collate_fn","repo":"orangesodahub/scenecraft","repo_kind":"official","path":"scenecraft/finetune/inference_controlnet_sd.py","file_url":"https://github.com/orangesodahub/scenecraft/blob/HEAD/scenecraft/finetune/inference_controlnet_sd.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"464e03da63a630e1"}},{"code_sha256_prefix":"9ff46a030ec6a83f","entry":"collate_fn","repo":"orangesodahub/scenecraft","repo_kind":"official","path":"scenecraft/renderer.py","file_url":"https://github.com/orangesodahub/scenecraft/blob/HEAD/scenecraft/renderer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9ff46a030ec6a83f"}},{"code_sha256_prefix":"dc6791e21ae7476b","entry":"find_save_id","repo":"OrangeSodahub/SceneCraft","repo_kind":"official","path":"scripts/generate_outputs.py","file_url":"https://github.com/OrangeSodahub/SceneCraft/blob/HEAD/scripts/generate_outputs.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dc6791e21ae7476b"}},{"code_sha256_prefix":"123e29ce80a5664d","entry":"hash_prompt","repo":"orangesodahub/scenecraft","repo_kind":"official","path":"scenecraft/prompt_processor.py","file_url":"https://github.com/orangesodahub/scenecraft/blob/HEAD/scenecraft/prompt_processor.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"123e29ce80a5664d"}},{"code_sha256_prefix":"bd5a05ce886da4ce","entry":"image_grid","repo":"orangesodahub/scenecraft","repo_kind":"official","path":"scenecraft/finetune/train_controlnet_sd.py","file_url":"https://github.com/orangesodahub/scenecraft/blob/HEAD/scenecraft/finetune/train_controlnet_sd.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bd5a05ce886da4ce"}},{"code_sha256_prefix":"694db9e4ce7de246","entry":"rank_zero_only","repo":"orangesodahub/scenecraft","repo_kind":"official","path":"scenecraft/utils.py","file_url":"https://github.com/orangesodahub/scenecraft/blob/HEAD/scenecraft/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"694db9e4ce7de246"}},{"code_sha256_prefix":"6efb3134e6a5d59d","entry":"zvar_loss","repo":"orangesodahub/scenecraft","repo_kind":"official","path":"scenecraft/model.py","file_url":"https://github.com/orangesodahub/scenecraft/blob/HEAD/scenecraft/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6efb3134e6a5d59d"}},{"code_sha256_prefix":"2f00b39db5a81466","entry":"import_model_class_from_model_name_or_path","repo":"orangesodahub/scenecraft","repo_kind":"official","path":"scenecraft/finetune/train_controlnet_sd.py","file_url":"https://github.com/orangesodahub/scenecraft/blob/HEAD/scenecraft/finetune/train_controlnet_sd.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2f00b39db5a81466"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}