{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/occdepth-a-depth-aware-method-for-3d-semantic","title":"OccDepth: A Depth-Aware Method for 3D Semantic Scene Completion","arxiv_id":"2302.13540","date":"2023-02-27","proceeding":null,"authors":["Ruihang Miao","Weizhou Liu","Mingrui Chen","Zheng Gong","Weixin Xu","Chen Hu","Shuchang Zhou"],"abstract":"3D Semantic Scene Completion (SSC) can provide dense geometric and semantic scene representations, which can be applied in the field of autonomous driving and robotic systems. It is challenging to estimate the complete geometry and semantics of a scene solely from visual images, and accurate depth information is crucial for restoring 3D geometry. In this paper, we propose the first stereo SSC method named OccDepth, which fully exploits implicit depth information from stereo images (or RGBD images) to help the recovery of 3D geometric structures. The Stereo Soft Feature Assignment (Stereo-SFA) module is proposed to better fuse 3D depth-aware features by implicitly learning the correlation between stereo images. In particular, when the input are RGBD image, a virtual stereo images can be generated through original RGB image and depth map. Besides, the Occupancy Aware Depth (OAD) module is used to obtain geometry-aware 3D features by knowledge distillation using pre-trained depth models. In addition, a reformed TartanAir benchmark, named SemanticTartanAir, is provided in this paper for further testing our OccDepth method on SSC task. Compared with the state-of-the-art RGB-inferred SSC method, extensive experiments on SemanticKITTI show that our OccDepth method achieves superior performance with improving +4.82% mIoU, of which +2.49% mIoU comes from stereo images and +2.33% mIoU comes from our proposed depth-aware method. Our code and trained models are available at https://github.com/megvii-research/OccDepth.","url_abs":"https://arxiv.org/abs/2302.13540v1","url_pdf":"https://arxiv.org/pdf/2302.13540v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"occdepth-a-depth-aware-method-for-3d-semantic","repo_url":"https://github.com/megvii-research/occdepth","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"3d-semantic-scene-completion","task_name":"3D Semantic Scene Completion"},{"task_slug":"3d-geometry","task_name":"3D geometry"},{"task_slug":"autonomous-driving","task_name":"Autonomous Driving"},{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"}],"methods":[{"method_slug":"aware","method_name":"AWARE"},{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-semantic-scene-completion-on-nyuv2","task":"3D Semantic Scene Completion","dataset":"NYUv2","model":"OccDepth","rank_in_archive_order":18,"of":28,"metrics":{"mIoU":"30.9"},"uses_additional_data":false},{"leaderboard":"/sota/3d-semantic-scene-completion-on-semantickitti","task":"3D Semantic Scene Completion","dataset":"SemanticKITTI","model":"OccDepth(RGB input only)","rank_in_archive_order":15,"of":20,"metrics":{"mIoU":"15.9"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2302.13540","atlas_url":"https://app.syntology.ai/?focus=2302.13540","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2302.13540"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/megvii-research/occdepth","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":3,"unverified":3},"by_repo_kind":{"official":{"samples":6,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5530781a69d17464","entry":"compute_super_CP_multilabel_loss","repo":"megvii-research/occdepth","repo_kind":"official","path":"occdepth/loss/CRP_loss.py","file_url":"https://github.com/megvii-research/occdepth/blob/HEAD/occdepth/loss/CRP_loss.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5530781a69d17464"}},{"code_sha256_prefix":"251f4945130ffbc3","entry":"get_iou","repo":"megvii-research/occdepth","repo_kind":"official","path":"occdepth/loss/sscMetrics.py","file_url":"https://github.com/megvii-research/occdepth/blob/HEAD/occdepth/loss/sscMetrics.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"251f4945130ffbc3"}},{"code_sha256_prefix":"d062b8e225df0d31","entry":"sem_scal_loss","repo":"megvii-research/occdepth","repo_kind":"official","path":"occdepth/loss/ssc_loss.py","file_url":"https://github.com/megvii-research/occdepth/blob/HEAD/occdepth/loss/ssc_loss.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d062b8e225df0d31"}},{"code_sha256_prefix":"033da7a511cc1166","entry":"KL_sep","repo":"megvii-research/occdepth","repo_kind":"official","path":"occdepth/loss/ssc_loss.py","file_url":"https://github.com/megvii-research/occdepth/blob/HEAD/occdepth/loss/ssc_loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"033da7a511cc1166"}},{"code_sha256_prefix":"1cf8e8cd991e96bb","entry":"geo_scal_loss","repo":"megvii-research/occdepth","repo_kind":"official","path":"occdepth/loss/ssc_loss.py","file_url":"https://github.com/megvii-research/occdepth/blob/HEAD/occdepth/loss/ssc_loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1cf8e8cd991e96bb"}},{"code_sha256_prefix":"40d8d0dfd5686afe","entry":"get_accuracy","repo":"megvii-research/occdepth","repo_kind":"official","path":"occdepth/loss/sscMetrics.py","file_url":"https://github.com/megvii-research/occdepth/blob/HEAD/occdepth/loss/sscMetrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"40d8d0dfd5686afe"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}