{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/3d-sketch-aware-semantic-scene-completion-via","title":"3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior","arxiv_id":"2003.14052","date":"2020-03-31","proceeding":"CVPR 2020 6","authors":["Xiaokang Chen","Kwan-Yee Lin","Chen Qian","Gang Zeng","Hongsheng Li"],"abstract":"The goal of the Semantic Scene Completion (SSC) task is to simultaneously predict a completed 3D voxel representation of volumetric occupancy and semantic labels of objects in the scene from a single-view observation. Since the computational cost generally increases explosively along with the growth of voxel resolution, most current state-of-the-arts have to tailor their framework into a low-resolution representation with the sacrifice of detail prediction. Thus, voxel resolution becomes one of the crucial difficulties that lead to the performance bottleneck. In this paper, we propose to devise a new geometry-based strategy to embed depth information with low-resolution voxel representation, which could still be able to encode sufficient geometric information, e.g., room layout, object's sizes and shapes, to infer the invisible areas of the scene with well structure-preserving details. To this end, we first propose a novel 3D sketch-aware feature embedding to explicitly encode geometric information effectively and efficiently. With the 3D sketch in hand, we further devise a simple yet effective semantic scene completion framework that incorporates a light-weight 3D Sketch Hallucination module to guide the inference of occupancy and the semantic labels via a semi-supervised structure prior learning strategy. We demonstrate that our proposed geometric embedding works better than the depth feature learning from habitual SSC frameworks. Our final model surpasses state-of-the-arts consistently on three public benchmarks, which only requires 3D volumes of 60 x 36 x 60 resolution for both input and output. The code and the supplementary material will be available at https://charlesCXK.github.io.","url_abs":"https://arxiv.org/abs/2003.14052v1","url_pdf":"https://arxiv.org/pdf/2003.14052v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"3d-sketch-aware-semantic-scene-completion-via","repo_url":"https://github.com/charlesCXK/3D-SketchAware-SSC","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"3d-sketch-aware-semantic-scene-completion-via","repo_url":"https://github.com/charlesCXK/TorchSSC","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"3d-semantic-scene-completion","task_name":"3D Semantic Scene Completion"},{"task_slug":"3d-semantic-scene-completion-from-a-single","task_name":"3D Semantic Scene Completion from a single RGB image"},{"task_slug":"hallucination","task_name":"Hallucination"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-semantic-scene-completion-on-nyuv2","task":"3D Semantic Scene Completion","dataset":"NYUv2","model":"3DSketch","rank_in_archive_order":7,"of":28,"metrics":{"mIoU":"41.1"},"uses_additional_data":false},{"leaderboard":"/sota/3d-semantic-scene-completion-from-a-single","task":"3D Semantic Scene Completion from a single RGB image","dataset":"NYUv2","model":"3DSketch (rgb input - reported in MonoScene paper)","rank_in_archive_order":4,"of":6,"metrics":{"mIoU":"22.91"},"uses_additional_data":false},{"leaderboard":"/sota/3d-semantic-scene-completion-from-a-single-1","task":"3D Semantic Scene Completion from a single RGB image","dataset":"SemanticKITTI","model":"3DSketch (rgb input - reported in MonoScene paper)","rank_in_archive_order":9,"of":9,"metrics":{"mIoU":"6.23"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2003.14052","atlas_url":"https://app.syntology.ai/?focus=2003.14052","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2003.14052"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/charlesCXK/TorchSSC","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/charlesCXK/3D-SketchAware-SSC","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"ran":2},"by_repo_kind":{"listed":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"fac5364e2f53c6db","entry":"conv3x3","repo":"charlesCXK/3D-SketchAware-SSC","repo_kind":"listed","path":"furnace/base_model/resnet.py","file_url":"https://github.com/charlesCXK/3D-SketchAware-SSC/blob/HEAD/furnace/base_model/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fac5364e2f53c6db"}},{"code_sha256_prefix":"bcd1f6e6a754daa8","entry":"get_resnet101","repo":"charlesCXK/3D-SketchAware-SSC","repo_kind":"listed","path":"model/sketch.nyu/resnet.py","file_url":"https://github.com/charlesCXK/3D-SketchAware-SSC/blob/HEAD/model/sketch.nyu/resnet.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bcd1f6e6a754daa8"}},{"code_sha256_prefix":"bdd94eaab631a1c1","entry":"get_resnet50","repo":"charlesCXK/3D-SketchAware-SSC","repo_kind":"listed","path":"model/sketch.nyu/resnet.py","file_url":"https://github.com/charlesCXK/3D-SketchAware-SSC/blob/HEAD/model/sketch.nyu/resnet.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bdd94eaab631a1c1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}