{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sampro3d-locating-sam-prompts-in-3d-for-zero","title":"SAMPro3D: Locating SAM Prompts in 3D for Zero-Shot Scene Segmentation","arxiv_id":"2311.17707","date":"2023-11-29","proceeding":null,"authors":["Mutian Xu","Xingyilang Yin","Lingteng Qiu","Yang Liu","Xin Tong","Xiaoguang Han"],"abstract":"We introduce SAMPro3D for zero-shot 3D indoor scene segmentation. Given the 3D point cloud and multiple posed 2D frames of 3D scenes, our approach segments 3D scenes by applying the pretrained Segment Anything Model (SAM) to 2D frames. Our key idea involves locating 3D points in scenes as natural 3D prompts to align their projected pixel prompts across frames, ensuring frame-consistency in both pixel prompts and their SAM-predicted masks. Moreover, we suggest filtering out low-quality 3D prompts based on feedback from all 2D frames, for enhancing segmentation quality. We also propose to consolidate different 3D prompts if they are segmenting the same object, bringing a more comprehensive segmentation. Notably, our method does not require any additional training on domain-specific data, enabling us to preserve the zero-shot power of SAM. Extensive qualitative and quantitative results show that our method consistently achieves higher quality and more diverse segmentation than previous zero-shot or fully supervised approaches, and in many cases even surpasses human-level annotations. The project page can be accessed at https://mutianxu.github.io/sampro3d/.","url_abs":"https://arxiv.org/abs/2311.17707v1","url_pdf":"https://arxiv.org/pdf/2311.17707v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sampro3d-locating-sam-prompts-in-3d-for-zero","repo_url":"https://github.com/GAP-LAB-CUHK-SZ/SAMPro3D","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"scene-segmentation","task_name":"Scene Segmentation"},{"task_slug":"scene-understanding","task_name":"Scene Understanding"},{"task_slug":"segmentation","task_name":"Segmentation"}],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"sam","method_name":"SAM"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2311.17707","atlas_url":"https://app.syntology.ai/?focus=2311.17707","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.17707"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/GAP-LAB-CUHK-SZ/SAMPro3D","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":12,"unverified":4},"by_repo_kind":{"official":{"samples":16,"ran":12,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c58cf40ab0e3d808","entry":"adjust_intrinsic","repo":"GAP-LAB-CUHK-SZ/SAMPro3D","repo_kind":"official","path":"dataset_preprocess/preprocess_util.py","file_url":"https://github.com/GAP-LAB-CUHK-SZ/SAMPro3D/blob/HEAD/dataset_preprocess/preprocess_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c58cf40ab0e3d808"}},{"code_sha256_prefix":"96acf36f2bf2b19d","entry":"apply_coords_torch","repo":"GAP-LAB-CUHK-SZ/SAMPro3D","repo_kind":"official","path":"utils/sam_utils.py","file_url":"https://github.com/GAP-LAB-CUHK-SZ/SAMPro3D/blob/HEAD/utils/sam_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"96acf36f2bf2b19d"}},{"code_sha256_prefix":"715832a79a564641","entry":"area_from_rle","repo":"GAP-LAB-CUHK-SZ/SAMPro3D","repo_kind":"official","path":"utils/sam_utils.py","file_url":"https://github.com/GAP-LAB-CUHK-SZ/SAMPro3D/blob/HEAD/utils/sam_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"715832a79a564641"}},{"code_sha256_prefix":"7b2ed44d3cabdb55","entry":"cal_iou","repo":"GAP-LAB-CUHK-SZ/SAMPro3D","repo_kind":"official","path":"utils/vis_utils.py","file_url":"https://github.com/GAP-LAB-CUHK-SZ/SAMPro3D/blob/HEAD/utils/vis_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7b2ed44d3cabdb55"}},{"code_sha256_prefix":"0288cbb68dfd84b3","entry":"compute_mapping","repo":"GAP-LAB-CUHK-SZ/SAMPro3D","repo_kind":"official","path":"utils/main_utils.py","file_url":"https://github.com/GAP-LAB-CUHK-SZ/SAMPro3D/blob/HEAD/utils/main_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0288cbb68dfd84b3"}},{"code_sha256_prefix":"16d72a87054ccef6","entry":"grouping","repo":"GAP-LAB-CUHK-SZ/SAMPro3D","repo_kind":"official","path":"libs/pointops/functions/grouping.py","file_url":"https://github.com/GAP-LAB-CUHK-SZ/SAMPro3D/blob/HEAD/libs/pointops/functions/grouping.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"16d72a87054ccef6"}},{"code_sha256_prefix":"5d9ad393128524ec","entry":"load_ply","repo":"GAP-LAB-CUHK-SZ/SAMPro3D","repo_kind":"official","path":"utils/main_utils.py","file_url":"https://github.com/GAP-LAB-CUHK-SZ/SAMPro3D/blob/HEAD/utils/main_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5d9ad393128524ec"}},{"code_sha256_prefix":"6385efe2f5a14a24","entry":"make_intrinsic","repo":"GAP-LAB-CUHK-SZ/SAMPro3D","repo_kind":"official","path":"dataset_preprocess/preprocess_util.py","file_url":"https://github.com/GAP-LAB-CUHK-SZ/SAMPro3D/blob/HEAD/dataset_preprocess/preprocess_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6385efe2f5a14a24"}},{"code_sha256_prefix":"f39dd6d6d882fdc3","entry":"map_label_image","repo":"GAP-LAB-CUHK-SZ/SAMPro3D","repo_kind":"official","path":"dataset_preprocess/preprocess_2d_scannet.py","file_url":"https://github.com/GAP-LAB-CUHK-SZ/SAMPro3D/blob/HEAD/dataset_preprocess/preprocess_2d_scannet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f39dd6d6d882fdc3"}},{"code_sha256_prefix":"38550edff36aa8dd","entry":"read_label_mapping","repo":"GAP-LAB-CUHK-SZ/SAMPro3D","repo_kind":"official","path":"dataset_preprocess/preprocess_2d_scannet.py","file_url":"https://github.com/GAP-LAB-CUHK-SZ/SAMPro3D/blob/HEAD/dataset_preprocess/preprocess_2d_scannet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"38550edff36aa8dd"}},{"code_sha256_prefix":"945af43d6054ff01","entry":"represents_int","repo":"GAP-LAB-CUHK-SZ/SAMPro3D","repo_kind":"official","path":"dataset_preprocess/preprocess_2d_scannet.py","file_url":"https://github.com/GAP-LAB-CUHK-SZ/SAMPro3D/blob/HEAD/dataset_preprocess/preprocess_2d_scannet.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"945af43d6054ff01"}},{"code_sha256_prefix":"62a58cd9338799e5","entry":"transform_pt_depth_scannet_torch","repo":"GAP-LAB-CUHK-SZ/SAMPro3D","repo_kind":"official","path":"utils/main_utils.py","file_url":"https://github.com/GAP-LAB-CUHK-SZ/SAMPro3D/blob/HEAD/utils/main_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"62a58cd9338799e5"}},{"code_sha256_prefix":"7069d6e0ba7c409e","entry":"box_xyxy_to_xywh","repo":"GAP-LAB-CUHK-SZ/SAMPro3D","repo_kind":"official","path":"utils/sam_utils.py","file_url":"https://github.com/GAP-LAB-CUHK-SZ/SAMPro3D/blob/HEAD/utils/sam_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7069d6e0ba7c409e"}},{"code_sha256_prefix":"60bae3c637ca79fb","entry":"interpolation","repo":"GAP-LAB-CUHK-SZ/SAMPro3D","repo_kind":"official","path":"libs/pointops/functions/interpolation.py","file_url":"https://github.com/GAP-LAB-CUHK-SZ/SAMPro3D/blob/HEAD/libs/pointops/functions/interpolation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"60bae3c637ca79fb"}},{"code_sha256_prefix":"ee1cc0a38944cca5","entry":"show_iou","repo":"GAP-LAB-CUHK-SZ/SAMPro3D","repo_kind":"official","path":"utils/vis_utils.py","file_url":"https://github.com/GAP-LAB-CUHK-SZ/SAMPro3D/blob/HEAD/utils/vis_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ee1cc0a38944cca5"}},{"code_sha256_prefix":"b3917ab72449e04c","entry":"show_mask","repo":"GAP-LAB-CUHK-SZ/SAMPro3D","repo_kind":"official","path":"utils/vis_utils.py","file_url":"https://github.com/GAP-LAB-CUHK-SZ/SAMPro3D/blob/HEAD/utils/vis_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b3917ab72449e04c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}