{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mixed-diffusion-for-3d-indoor-scene-synthesis","title":"Mixed Diffusion for 3D Indoor Scene Synthesis","arxiv_id":"2405.21066","date":"2024-05-31","proceeding":null,"authors":["Siyi Hu","Diego Martin Arroyo","Stephanie Debats","Fabian Manhardt","Luca Carlone","Federico Tombari"],"abstract":"Generating realistic 3D scenes is an area of growing interest in computer vision and robotics. However, creating high-quality, diverse synthetic 3D content often requires expert intervention, making it costly and complex. Recently, efforts to automate this process with learning techniques, particularly diffusion models, have shown significant improvements in tasks like furniture rearrangement. However, applying diffusion models to floor-conditioned indoor scene synthesis remains under-explored. This task is especially challenging as it requires arranging objects in continuous space while selecting from discrete object categories, posing unique difficulties for conventional diffusion methods. To bridge this gap, we present MiDiffusion, a novel mixed discrete-continuous diffusion model designed to synthesize plausible 3D indoor scenes given a floor plan and pre-arranged objects. We represent a scene layout by a 2D floor plan and a set of objects, each defined by category, location, size, and orientation. Our approach uniquely applies structured corruption across mixed discrete semantic and continuous geometric domains, resulting in a better-conditioned problem for denoising. Evaluated on the 3D-FRONT dataset, MiDiffusion outperforms state-of-the-art autoregressive and diffusion models in floor-conditioned 3D scene synthesis. Additionally, it effectively handles partial object constraints via a corruption-and-masking strategy without task-specific training, demonstrating advantages in scene completion and furniture arrangement tasks.","url_abs":"https://arxiv.org/abs/2405.21066v2","url_pdf":"https://arxiv.org/pdf/2405.21066v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mixed-diffusion-for-3d-indoor-scene-synthesis","repo_url":"https://github.com/mit-spark/midiffusion","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"indoor-scene-synthesis","task_name":"Indoor Scene Synthesis"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2405.21066","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.21066"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mit-spark/midiffusion","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran":6,"ran_honours":1,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":10,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"5e963c0dc8a616e5","entry":"axis_aligned_bbox_overlaps_3d","repo":"mit-spark/midiffusion","repo_kind":"official","path":"midiffusion/networks/loss.py","file_url":"https://github.com/mit-spark/midiffusion/blob/HEAD/midiffusion/networks/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"5e963c0dc8a616e5"}},{"code_sha256_prefix":"190bebc842613fd1","entry":"extract_params","repo":"mit-spark/midiffusion","repo_kind":"official","path":"midiffusion/networks/diffusion_mixed.py","file_url":"https://github.com/mit-spark/midiffusion/blob/HEAD/midiffusion/networks/diffusion_mixed.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"190bebc842613fd1"}},{"code_sha256_prefix":"00edd28ef6f548e0","entry":"freeze_network","repo":"mit-spark/midiffusion","repo_kind":"official","path":"midiffusion/networks/frozen_batchnorm.py","file_url":"https://github.com/mit-spark/midiffusion/blob/HEAD/midiffusion/networks/frozen_batchnorm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"00edd28ef6f548e0"}},{"code_sha256_prefix":"ea8ccee07ccf89b1","entry":"get_betas","repo":"mit-spark/midiffusion","repo_kind":"official","path":"midiffusion/networks/diffusion_ddpm.py","file_url":"https://github.com/mit-spark/midiffusion/blob/HEAD/midiffusion/networks/diffusion_ddpm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"ea8ccee07ccf89b1"}},{"code_sha256_prefix":"9b1c4e6a2ce41fff","entry":"get_feature_mask","repo":"mit-spark/midiffusion","repo_kind":"official","path":"midiffusion/evaluation/utils.py","file_url":"https://github.com/mit-spark/midiffusion/blob/HEAD/midiffusion/evaluation/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"9b1c4e6a2ce41fff"}},{"code_sha256_prefix":"e532cc2df7cd402f","entry":"log_add_exp","repo":"mit-spark/midiffusion","repo_kind":"official","path":"midiffusion/networks/diffusion_d3pm.py","file_url":"https://github.com/mit-spark/midiffusion/blob/HEAD/midiffusion/networks/diffusion_d3pm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"e532cc2df7cd402f"}},{"code_sha256_prefix":"f4b18dd8f67674cb","entry":"log_categorical","repo":"mit-spark/midiffusion","repo_kind":"official","path":"midiffusion/networks/diffusion_d3pm.py","file_url":"https://github.com/mit-spark/midiffusion/blob/HEAD/midiffusion/networks/diffusion_d3pm.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f4b18dd8f67674cb"}},{"code_sha256_prefix":"ec4a40e4187ccdac","entry":"normal_kl","repo":"mit-spark/midiffusion","repo_kind":"official","path":"midiffusion/networks/diffusion_ddpm.py","file_url":"https://github.com/mit-spark/midiffusion/blob/HEAD/midiffusion/networks/diffusion_ddpm.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ec4a40e4187ccdac"}},{"code_sha256_prefix":"5d0cae21e2c438ad","entry":"get_feature_extractor","repo":"mit-spark/midiffusion","repo_kind":"official","path":"midiffusion/networks/feature_extractors.py","file_url":"https://github.com/mit-spark/midiffusion/blob/HEAD/midiffusion/networks/feature_extractors.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"5d0cae21e2c438ad"}},{"code_sha256_prefix":"f7bc6629510abc28","entry":"log_1_min_a","repo":"mit-spark/midiffusion","repo_kind":"official","path":"midiffusion/networks/diffusion_d3pm.py","file_url":"https://github.com/mit-spark/midiffusion/blob/HEAD/midiffusion/networks/diffusion_d3pm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"f7bc6629510abc28"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}