{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/surgicalsam-efficient-class-promptable","title":"SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentation","arxiv_id":"2308.08746","date":"2023-08-17","proceeding":null,"authors":["Wenxi Yue","Jing Zhang","Kun Hu","Yong Xia","Jiebo Luo","Zhiyong Wang"],"abstract":"The Segment Anything Model (SAM) is a powerful foundation model that has revolutionised image segmentation. To apply SAM to surgical instrument segmentation, a common approach is to locate precise points or boxes of instruments and then use them as prompts for SAM in a zero-shot manner. However, we observe two problems with this naive pipeline: (1) the domain gap between natural objects and surgical instruments leads to inferior generalisation of SAM; and (2) SAM relies on precise point or box locations for accurate segmentation, requiring either extensive manual guidance or a well-performing specialist detector for prompt preparation, which leads to a complex multi-stage pipeline. To address these problems, we introduce SurgicalSAM, a novel end-to-end efficient-tuning approach for SAM to effectively integrate surgical-specific information with SAM's pre-trained knowledge for improved generalisation. Specifically, we propose a lightweight prototype-based class prompt encoder for tuning, which directly generates prompt embeddings from class prototypes and eliminates the use of explicit prompts for improved robustness and a simpler pipeline. In addition, to address the low inter-class variance among surgical instrument categories, we propose contrastive prototype learning, further enhancing the discrimination of the class prototypes for more accurate class prompting. The results of extensive experiments on both EndoVis2018 and EndoVis2017 datasets demonstrate that SurgicalSAM achieves state-of-the-art performance while only requiring a small number of tunable parameters. The source code is available at https://github.com/wenxi-yue/SurgicalSAM.","url_abs":"https://arxiv.org/abs/2308.08746v2","url_pdf":"https://arxiv.org/pdf/2308.08746v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"surgicalsam-efficient-class-promptable","repo_url":"https://github.com/wenxi-yue/surgicalsam","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-segmentation","task_name":"Image Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[{"method_slug":"sam","method_name":"SAM"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2308.08746","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.08746"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wenxi-yue/surgicalsam","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":7,"ran_fixture":3,"unverified":4},"by_repo_kind":{"official":{"samples":14,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c415b7a608d4e28f","entry":"create_binary_masks","repo":"wenxi-yue/surgicalsam","repo_kind":"official","path":"surgicalSAM/utils.py","file_url":"https://github.com/wenxi-yue/surgicalsam/blob/HEAD/surgicalSAM/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c415b7a608d4e28f"}},{"code_sha256_prefix":"cc557c224482ee77","entry":"create_endovis_masks","repo":"wenxi-yue/surgicalsam","repo_kind":"official","path":"surgicalSAM/utils.py","file_url":"https://github.com/wenxi-yue/surgicalsam/blob/HEAD/surgicalSAM/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cc557c224482ee77"}},{"code_sha256_prefix":"27cddc06d6605b4e","entry":"dice_loss","repo":"wenxi-yue/surgicalsam","repo_kind":"official","path":"surgicalSAM/loss.py","file_url":"https://github.com/wenxi-yue/surgicalsam/blob/HEAD/surgicalSAM/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"27cddc06d6605b4e"}},{"code_sha256_prefix":"fd120c1f88221c37","entry":"eval_endovis","repo":"wenxi-yue/surgicalsam","repo_kind":"official","path":"surgicalSAM/utils.py","file_url":"https://github.com/wenxi-yue/surgicalsam/blob/HEAD/surgicalSAM/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fd120c1f88221c37"}},{"code_sha256_prefix":"733d7f0bedcb74c2","entry":"get_rel_pos","repo":"wenxi-yue/surgicalsam","repo_kind":"official","path":"segment_anything/modeling/image_encoder.py","file_url":"https://github.com/wenxi-yue/surgicalsam/blob/HEAD/segment_anything/modeling/image_encoder.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"733d7f0bedcb74c2"}},{"code_sha256_prefix":"45e6963aa726c950","entry":"model_forward_function","repo":"wenxi-yue/surgicalsam","repo_kind":"official","path":"surgicalSAM/model_forward.py","file_url":"https://github.com/wenxi-yue/surgicalsam/blob/HEAD/surgicalSAM/model_forward.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"45e6963aa726c950"}},{"code_sha256_prefix":"c3b872692385a7a1","entry":"postprocess_masks","repo":"wenxi-yue/surgicalsam","repo_kind":"official","path":"surgicalSAM/model_forward.py","file_url":"https://github.com/wenxi-yue/surgicalsam/blob/HEAD/surgicalSAM/model_forward.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c3b872692385a7a1"}},{"code_sha256_prefix":"b2b9a1966482ef96","entry":"reduce_loss","repo":"wenxi-yue/surgicalsam","repo_kind":"official","path":"surgicalSAM/loss.py","file_url":"https://github.com/wenxi-yue/surgicalsam/blob/HEAD/surgicalSAM/loss.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b2b9a1966482ef96"}},{"code_sha256_prefix":"bd54cbe9d9fb8b02","entry":"weight_reduce_loss","repo":"wenxi-yue/surgicalsam","repo_kind":"official","path":"surgicalSAM/loss.py","file_url":"https://github.com/wenxi-yue/surgicalsam/blob/HEAD/surgicalSAM/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bd54cbe9d9fb8b02"}},{"code_sha256_prefix":"105fa08885dc36cc","entry":"window_partition","repo":"wenxi-yue/surgicalsam","repo_kind":"official","path":"segment_anything/modeling/image_encoder.py","file_url":"https://github.com/wenxi-yue/surgicalsam/blob/HEAD/segment_anything/modeling/image_encoder.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"105fa08885dc36cc"}},{"code_sha256_prefix":"f9bc5f31ce61cee8","entry":"build_sam_vit_b","repo":"wenxi-yue/surgicalsam","repo_kind":"official","path":"segment_anything/build_sam.py","file_url":"https://github.com/wenxi-yue/surgicalsam/blob/HEAD/segment_anything/build_sam.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f9bc5f31ce61cee8"}},{"code_sha256_prefix":"6b77c1f11fff3ed6","entry":"build_sam_vit_h","repo":"wenxi-yue/surgicalsam","repo_kind":"official","path":"segment_anything/build_sam.py","file_url":"https://github.com/wenxi-yue/surgicalsam/blob/HEAD/segment_anything/build_sam.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6b77c1f11fff3ed6"}},{"code_sha256_prefix":"3f8890e695469246","entry":"build_sam_vit_l","repo":"wenxi-yue/surgicalsam","repo_kind":"official","path":"segment_anything/build_sam.py","file_url":"https://github.com/wenxi-yue/surgicalsam/blob/HEAD/segment_anything/build_sam.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3f8890e695469246"}},{"code_sha256_prefix":"27be441cc8213e52","entry":"window_unpartition","repo":"wenxi-yue/surgicalsam","repo_kind":"official","path":"segment_anything/modeling/image_encoder.py","file_url":"https://github.com/wenxi-yue/surgicalsam/blob/HEAD/segment_anything/modeling/image_encoder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"27be441cc8213e52"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}