{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/semcity-semantic-scene-generation-with","title":"SemCity: Semantic Scene Generation with Triplane Diffusion","arxiv_id":"2403.07773","date":"2024-03-12","proceeding":"CVPR 2024 1","authors":["Jumin Lee","Sebin Lee","Changho Jo","Woobin Im","Juhyeong Seon","Sung-Eui Yoon"],"abstract":"We present \"SemCity,\" a 3D diffusion model for semantic scene generation in real-world outdoor environments. Most 3D diffusion models focus on generating a single object, synthetic indoor scenes, or synthetic outdoor scenes, while the generation of real-world outdoor scenes is rarely addressed. In this paper, we concentrate on generating a real-outdoor scene through learning a diffusion model on a real-world outdoor dataset. In contrast to synthetic data, real-outdoor datasets often contain more empty spaces due to sensor limitations, causing challenges in learning real-outdoor distributions. To address this issue, we exploit a triplane representation as a proxy form of scene distributions to be learned by our diffusion model. Furthermore, we propose a triplane manipulation that integrates seamlessly with our triplane diffusion model. The manipulation improves our diffusion model's applicability in a variety of downstream tasks related to outdoor scene generation such as scene inpainting, scene outpainting, and semantic scene completion refinements. In experimental results, we demonstrate that our triplane diffusion model shows meaningful generation results compared with existing work in a real-outdoor dataset, SemanticKITTI. We also show our triplane manipulation facilitates seamlessly adding, removing, or modifying objects within a scene. Further, it also enables the expansion of scenes toward a city-level scale. Finally, we evaluate our method on semantic scene completion refinements where our diffusion model enhances predictions of semantic scene completion networks by learning scene distribution. Our code is available at https://github.com/zoomin-lee/SemCity.","url_abs":"https://arxiv.org/abs/2403.07773v3","url_pdf":"https://arxiv.org/pdf/2403.07773v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"semcity-semantic-scene-generation-with","repo_url":"https://github.com/zoomin-lee/semcity","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"scene-generation","task_name":"Scene Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2403.07773","atlas_url":"https://app.syntology.ai/?focus=2403.07773","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.07773"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zoomin-lee/semcity","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/zoomin-lee/SemCity","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":11,"ran_draft_wrong":2,"ran_honours":3,"ran_fixture":1,"unverified":5},"by_repo_kind":{"official":{"samples":22,"ran":17,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b3036001ee3ed95e","entry":"TriplaneConv","repo":"zoomin-lee/SemCity","repo_kind":"official","path":"diffusion/unet_triplane.py","file_url":"https://github.com/zoomin-lee/SemCity/blob/HEAD/diffusion/unet_triplane.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b3036001ee3ed95e"}},{"code_sha256_prefix":"ff17b7bc4046558e","entry":"TriplaneDownsample2x","repo":"zoomin-lee/SemCity","repo_kind":"official","path":"diffusion/unet_triplane.py","file_url":"https://github.com/zoomin-lee/SemCity/blob/HEAD/diffusion/unet_triplane.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ff17b7bc4046558e"}},{"code_sha256_prefix":"98c1fed464e645bb","entry":"TriplaneUpsample2x","repo":"zoomin-lee/SemCity","repo_kind":"official","path":"diffusion/unet_triplane.py","file_url":"https://github.com/zoomin-lee/SemCity/blob/HEAD/diffusion/unet_triplane.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"98c1fed464e645bb"}},{"code_sha256_prefix":"cfd76fd0d89574a4","entry":"approx_standard_normal_cdf","repo":"zoomin-lee/semcity","repo_kind":"official","path":"diffusion/losses.py","file_url":"https://github.com/zoomin-lee/semcity/blob/HEAD/diffusion/losses.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cfd76fd0d89574a4"}},{"code_sha256_prefix":"2ab2316ac6fdd869","entry":"betas_for_alpha_bar","repo":"zoomin-lee/semcity","repo_kind":"official","path":"diffusion/gaussian_diffusion.py","file_url":"https://github.com/zoomin-lee/semcity/blob/HEAD/diffusion/gaussian_diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2ab2316ac6fdd869"}},{"code_sha256_prefix":"87ef5d1a6cbdb58b","entry":"compute_tdf","repo":"zoomin-lee/semcity","repo_kind":"official","path":"dataset/kitti_dataset.py","file_url":"https://github.com/zoomin-lee/semcity/blob/HEAD/dataset/kitti_dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"87ef5d1a6cbdb58b"}},{"code_sha256_prefix":"2c048a12482e6f52","entry":"decompose_featmaps","repo":"zoomin-lee/SemCity","repo_kind":"official","path":"diffusion/unet_triplane.py","file_url":"https://github.com/zoomin-lee/SemCity/blob/HEAD/diffusion/unet_triplane.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2c048a12482e6f52"}},{"code_sha256_prefix":"cd33283d615fb3d7","entry":"discretized_gaussian_log_likelihood","repo":"zoomin-lee/semcity","repo_kind":"official","path":"diffusion/losses.py","file_url":"https://github.com/zoomin-lee/semcity/blob/HEAD/diffusion/losses.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cd33283d615fb3d7"}},{"code_sha256_prefix":"dda2d38f31cffc71","entry":"flip","repo":"zoomin-lee/semcity","repo_kind":"official","path":"dataset/kitti_dataset.py","file_url":"https://github.com/zoomin-lee/semcity/blob/HEAD/dataset/kitti_dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dda2d38f31cffc71"}},{"code_sha256_prefix":"a086d6286a40b889","entry":"get_named_beta_schedule","repo":"zoomin-lee/semcity","repo_kind":"official","path":"diffusion/gaussian_diffusion.py","file_url":"https://github.com/zoomin-lee/semcity/blob/HEAD/diffusion/gaussian_diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a086d6286a40b889"}},{"code_sha256_prefix":"e41367ad14ff58fd","entry":"get_param_groups_and_shapes","repo":"zoomin-lee/semcity","repo_kind":"official","path":"diffusion/fp16_util.py","file_url":"https://github.com/zoomin-lee/semcity/blob/HEAD/diffusion/fp16_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e41367ad14ff58fd"}},{"code_sha256_prefix":"9a009f7bfad56ea2","entry":"get_query","repo":"zoomin-lee/semcity","repo_kind":"official","path":"dataset/kitti_dataset.py","file_url":"https://github.com/zoomin-lee/semcity/blob/HEAD/dataset/kitti_dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a009f7bfad56ea2"}},{"code_sha256_prefix":"e20dd5102da3b050","entry":"make_master_params","repo":"zoomin-lee/semcity","repo_kind":"official","path":"diffusion/fp16_util.py","file_url":"https://github.com/zoomin-lee/semcity/blob/HEAD/diffusion/fp16_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e20dd5102da3b050"}},{"code_sha256_prefix":"bcd8b4acab199405","entry":"make_output_format","repo":"zoomin-lee/semcity","repo_kind":"official","path":"diffusion/logger.py","file_url":"https://github.com/zoomin-lee/semcity/blob/HEAD/diffusion/logger.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bcd8b4acab199405"}},{"code_sha256_prefix":"892ae51921341521","entry":"mask_img","repo":"zoomin-lee/semcity","repo_kind":"official","path":"diffusion/nn.py","file_url":"https://github.com/zoomin-lee/semcity/blob/HEAD/diffusion/nn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"892ae51921341521"}},{"code_sha256_prefix":"cf2798b666b231ca","entry":"normal_kl","repo":"zoomin-lee/semcity","repo_kind":"official","path":"diffusion/losses.py","file_url":"https://github.com/zoomin-lee/semcity/blob/HEAD/diffusion/losses.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cf2798b666b231ca"}},{"code_sha256_prefix":"64fff1e30802b815","entry":"unflatten_master_params","repo":"zoomin-lee/semcity","repo_kind":"official","path":"diffusion/fp16_util.py","file_url":"https://github.com/zoomin-lee/semcity/blob/HEAD/diffusion/fp16_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"64fff1e30802b815"}},{"code_sha256_prefix":"fe0e214e83341048","entry":"TriplaneResBlock","repo":"zoomin-lee/SemCity","repo_kind":"official","path":"diffusion/unet_triplane.py","file_url":"https://github.com/zoomin-lee/SemCity/blob/HEAD/diffusion/unet_triplane.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fe0e214e83341048"}},{"code_sha256_prefix":"8e7981a79272ed82","entry":"TriplaneUNetModel","repo":"zoomin-lee/SemCity","repo_kind":"official","path":"diffusion/unet_triplane.py","file_url":"https://github.com/zoomin-lee/SemCity/blob/HEAD/diffusion/unet_triplane.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8e7981a79272ed82"}},{"code_sha256_prefix":"58f62059adb7a36e","entry":"compose_featmaps","repo":"zoomin-lee/SemCity","repo_kind":"official","path":"diffusion/unet_triplane.py","file_url":"https://github.com/zoomin-lee/SemCity/blob/HEAD/diffusion/unet_triplane.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"58f62059adb7a36e"}},{"code_sha256_prefix":"e515a67f7f32e76d","entry":"mpi_weighted_mean","repo":"zoomin-lee/semcity","repo_kind":"official","path":"diffusion/logger.py","file_url":"https://github.com/zoomin-lee/semcity/blob/HEAD/diffusion/logger.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e515a67f7f32e76d"}},{"code_sha256_prefix":"0c6607473a4c4c55","entry":"profile","repo":"zoomin-lee/semcity","repo_kind":"official","path":"diffusion/logger.py","file_url":"https://github.com/zoomin-lee/semcity/blob/HEAD/diffusion/logger.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0c6607473a4c4c55"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}