{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-a-comprehensive-efficient-and","title":"Towards a Comprehensive, Efficient and Promptable Anatomic Structure Segmentation Model using 3D Whole-body CT Scans","arxiv_id":"2403.15063","date":"2024-03-22","proceeding":null,"authors":["Heng Guo","Jianfeng Zhang","Jiaxing Huang","Tony C. W. Mok","Dazhou Guo","Ke Yan","Le Lu","Dakai Jin","Minfeng Xu"],"abstract":"Segment anything model (SAM) demonstrates strong generalization ability on natural image segmentation. However, its direct adaptation in medical image segmentation tasks shows significant performance drops. It also requires an excessive number of prompt points to obtain a reasonable accuracy. Although quite a few studies explore adapting SAM into medical image volumes, the efficiency of 2D adaptation methods is unsatisfactory and 3D adaptation methods are only capable of segmenting specific organs/tumors. In this work, we propose a comprehensive and scalable 3D SAM model for whole-body CT segmentation, named CT-SAM3D. Instead of adapting SAM, we propose a 3D promptable segmentation model using a (nearly) fully labeled CT dataset. To train CT-SAM3D effectively, ensuring the model's accurate responses to higher-dimensional spatial prompts is crucial, and 3D patch-wise training is required due to GPU memory constraints. Therefore, we propose two key technical developments: 1) a progressively and spatially aligned prompt encoding method to effectively encode click prompts in local 3D space; and 2) a cross-patch prompt scheme to capture more 3D spatial context, which is beneficial for reducing the editing workloads when interactively prompting on large organs. CT-SAM3D is trained using a curated dataset of 1204 CT scans containing 107 whole-body anatomies and extensively validated using five datasets, achieving significantly better results against all previous SAM-derived models. Code, data, and our 3D interactive segmentation tool with quasi-real-time responses are available at https://github.com/alibaba-damo-academy/ct-sam3d.","url_abs":"https://arxiv.org/abs/2403.15063v2","url_pdf":"https://arxiv.org/pdf/2403.15063v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-a-comprehensive-efficient-and","repo_url":"https://github.com/alibaba-damo-academy/ct-sam3d","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":null,"task_name":"GPU"},{"task_slug":"image-segmentation","task_name":"Image Segmentation"},{"task_slug":"interactive-segmentation","task_name":"Interactive Segmentation"},{"task_slug":"medical-image-segmentation","task_name":"Medical Image Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[{"method_slug":"sam","method_name":"SAM"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.15063","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.15063"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/alibaba-damo-academy/ct-sam3d","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":4,"unverified":6},"by_repo_kind":{"official":{"samples":10,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6a9e9980bf9e889d","entry":"gaussian3D","repo":"alibaba-damo-academy/ct-sam3d","repo_kind":"official","path":"ct_sam/utils/gaussian_target.py","file_url":"https://github.com/alibaba-damo-academy/ct-sam3d/blob/HEAD/ct_sam/utils/gaussian_target.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6a9e9980bf9e889d"}},{"code_sha256_prefix":"4ba1cdcf5b7124b3","entry":"get_local_maximum","repo":"alibaba-damo-academy/ct-sam3d","repo_kind":"official","path":"ct_sam/utils/gaussian_target.py","file_url":"https://github.com/alibaba-damo-academy/ct-sam3d/blob/HEAD/ct_sam/utils/gaussian_target.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4ba1cdcf5b7124b3"}},{"code_sha256_prefix":"84c7ce06c891d321","entry":"get_topk_from_heatmap","repo":"alibaba-damo-academy/ct-sam3d","repo_kind":"official","path":"ct_sam/utils/gaussian_target.py","file_url":"https://github.com/alibaba-damo-academy/ct-sam3d/blob/HEAD/ct_sam/utils/gaussian_target.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"84c7ce06c891d321"}},{"code_sha256_prefix":"b1c4fac402715a6f","entry":"partial_weight_update","repo":"alibaba-damo-academy/ct-sam3d","repo_kind":"official","path":"ct_sam/utils/network.py","file_url":"https://github.com/alibaba-damo-academy/ct-sam3d/blob/HEAD/ct_sam/utils/network.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b1c4fac402715a6f"}},{"code_sha256_prefix":"2f903a1d8b06697f","entry":"build_sam_vit_b","repo":"alibaba-damo-academy/ct-sam3d","repo_kind":"official","path":"ct_sam/builder.py","file_url":"https://github.com/alibaba-damo-academy/ct-sam3d/blob/HEAD/ct_sam/builder.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2f903a1d8b06697f"}},{"code_sha256_prefix":"4bfe3eb5a28153f2","entry":"build_sam_vit_h","repo":"alibaba-damo-academy/ct-sam3d","repo_kind":"official","path":"ct_sam/builder.py","file_url":"https://github.com/alibaba-damo-academy/ct-sam3d/blob/HEAD/ct_sam/builder.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4bfe3eb5a28153f2"}},{"code_sha256_prefix":"bdb954e979dda5a5","entry":"build_sam_vit_l","repo":"alibaba-damo-academy/ct-sam3d","repo_kind":"official","path":"ct_sam/builder.py","file_url":"https://github.com/alibaba-damo-academy/ct-sam3d/blob/HEAD/ct_sam/builder.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bdb954e979dda5a5"}},{"code_sha256_prefix":"6c539928d0946d16","entry":"restv2_base","repo":"alibaba-damo-academy/ct-sam3d","repo_kind":"official","path":"ct_sam/models/rest_v2.py","file_url":"https://github.com/alibaba-damo-academy/ct-sam3d/blob/HEAD/ct_sam/models/rest_v2.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6c539928d0946d16"}},{"code_sha256_prefix":"ac26a4c6c80c693c","entry":"restv2_small","repo":"alibaba-damo-academy/ct-sam3d","repo_kind":"official","path":"ct_sam/models/rest_v2.py","file_url":"https://github.com/alibaba-damo-academy/ct-sam3d/blob/HEAD/ct_sam/models/rest_v2.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ac26a4c6c80c693c"}},{"code_sha256_prefix":"0e77f0801a7185b2","entry":"restv2_tiny","repo":"alibaba-damo-academy/ct-sam3d","repo_kind":"official","path":"ct_sam/models/rest_v2.py","file_url":"https://github.com/alibaba-damo-academy/ct-sam3d/blob/HEAD/ct_sam/models/rest_v2.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0e77f0801a7185b2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}