{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/3d-sis-3d-semantic-instance-segmentation-of","title":"3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans","arxiv_id":"1812.07003","date":"2018-12-17","proceeding":"CVPR 2019 6","authors":["Ji Hou","Angela Dai","Matthias Nießner"],"abstract":"We introduce 3D-SIS, a novel neural network architecture for 3D semantic\ninstance segmentation in commodity RGB-D scans. The core idea of our method is\nto jointly learn from both geometric and color signal, thus enabling accurate\ninstance predictions. Rather than operate solely on 2D frames, we observe that\nmost computer vision applications have multi-view RGB-D input available, which\nwe leverage to construct an approach for 3D instance segmentation that\neffectively fuses together these multi-modal inputs. Our network leverages\nhigh-resolution RGB input by associating 2D images with the volumetric grid\nbased on the pose alignment of the 3D reconstruction. For each image, we first\nextract 2D features for each pixel with a series of 2D convolutions; we then\nbackproject the resulting feature vector to the associated voxel in the 3D\ngrid. This combination of 2D and 3D feature learning allows significantly\nhigher accuracy object detection and instance segmentation than\nstate-of-the-art alternatives. We show results on both synthetic and real-world\npublic benchmarks, achieving an improvement in mAP of over 13 on real-world\ndata.","url_abs":"http://arxiv.org/abs/1812.07003v3","url_pdf":"http://arxiv.org/pdf/1812.07003v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"3d-sis-3d-semantic-instance-segmentation-of","repo_url":"https://github.com/Sekunde/3D-SIS","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"3d-instance-segmentation-1","task_name":"3D Instance Segmentation"},{"task_slug":"3d-object-detection","task_name":"3D Object Detection"},{"task_slug":"3d-reconstruction","task_name":"3D Reconstruction"},{"task_slug":"3d-semantic-instance-segmentation","task_name":"3D Semantic Instance Segmentation"},{"task_slug":"instance-segmentation","task_name":"Instance Segmentation"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-instance-segmentation-on-scannetv2","task":"3D Instance Segmentation","dataset":"ScanNet(v2)","model":"3D-SIS","rank_in_archive_order":28,"of":32,"metrics":{"mAP @ 50":"38.2"},"uses_additional_data":false},{"leaderboard":"/sota/3d-object-detection-on-scannetv2","task":"3D Object Detection","dataset":"ScanNetV2","model":"3D-SIS","rank_in_archive_order":31,"of":33,"metrics":{"mAP@0.25":"40.2","mAP@0.5":"22.5"},"uses_additional_data":false},{"leaderboard":"/sota/3d-semantic-instance-segmentation-on-1","task":"3D Semantic Instance Segmentation","dataset":"ScanNetV2","model":"3D-SIS","rank_in_archive_order":3,"of":5,"metrics":{"mAP@0.50":"38.2"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1812.07003","atlas_url":"https://app.syntology.ai/?focus=1812.07003","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1812.07003"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Sekunde/3D-SIS","reach":null}],"summary":{"unverified":2},"by_repo_kind":{"official":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"65d7c27f9ea53043","entry":"write_bbox","repo":"Sekunde/3D-SIS","repo_kind":"official","path":"tools/visualization.py","file_url":"https://github.com/Sekunde/3D-SIS/blob/HEAD/tools/visualization.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"65d7c27f9ea53043"}},{"code_sha256_prefix":"99d4a7756297e4f8","entry":"write_mask_pointcloud","repo":"Sekunde/3D-SIS","repo_kind":"official","path":"tools/visualization.py","file_url":"https://github.com/Sekunde/3D-SIS/blob/HEAD/tools/visualization.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"99d4a7756297e4f8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}