{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/2408-00874","title":"Medical SAM 2: Segment medical images as video via Segment Anything Model 2","arxiv_id":"2408.00874","date":"2024-08-01","proceeding":null,"authors":["Jiayuan Zhu","Abdullah Hamdi","Yunli Qi","Yueming Jin","Junde Wu"],"abstract":"Medical image segmentation plays a pivotal role in clinical diagnostics and treatment planning, yet existing models often face challenges in generalization and in handling both 2D and 3D data uniformly. In this paper, we introduce Medical SAM 2 (MedSAM-2), a generalized auto-tracking model for universal 2D and 3D medical image segmentation. The core concept is to leverage the Segment Anything Model 2 (SAM2) pipeline to treat all 2D and 3D medical segmentation tasks as a video object tracking problem. To put it into practice, we propose a novel \\emph{self-sorting memory bank} mechanism that dynamically selects informative embeddings based on confidence and dissimilarity, regardless of temporal order. This mechanism not only significantly improves performance in 3D medical image segmentation but also unlocks a \\emph{One-Prompt Segmentation} capability for 2D images, allowing segmentation across multiple images from a single prompt without temporal relationships. We evaluated MedSAM-2 on five 2D tasks and nine 3D tasks, including white blood cells, optic cups, retinal vessels, mandibles, coronary arteries, kidney tumors, liver tumors, breast cancer, nasopharynx cancer, vestibular schwannoma, mediastinal lymph nodules, cerebral artery, inferior alveolar nerve, and abdominal organs, comparing it against state-of-the-art (SOTA) models in task-tailored, general and interactive segmentation settings. Our findings demonstrate that MedSAM-2 surpasses a wide range of existing models and updates new SOTA on several benchmarks. The code is released on the project page: https://supermedintel.github.io/Medical-SAM2/.","url_abs":"https://arxiv.org/abs/2408.00874v2","url_pdf":"https://arxiv.org/pdf/2408.00874v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"2408-00874","repo_url":"https://github.com/medicinetoken/medical-sam2","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"2408-00874","repo_url":"https://github.com/supermedintel/medical-sam2","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"image-segmentation","task_name":"Image Segmentation"},{"task_slug":"interactive-segmentation","task_name":"Interactive Segmentation"},{"task_slug":"medical-image-segmentation","task_name":"Medical Image Segmentation"},{"task_slug":"object-tracking","task_name":"Object Tracking"},{"task_slug":"philosophy","task_name":"Philosophy"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"video-object-tracking","task_name":"Video Object Tracking"}],"methods":[{"method_slug":"sam","method_name":"SAM"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2408.00874","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2408.00874"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/supermedintel/medical-sam2","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/medicinetoken/medical-sam2","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_violates":1,"ran_draft_wrong":2,"ran_honours":1,"ran_fixture":1,"ran":2,"unverified":4},"by_repo_kind":{"official":{"samples":11,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7ceaad42da42edd5","entry":"compute_axial_cis","repo":"medicinetoken/medical-sam2","repo_kind":"official","path":"sam2_train/modeling/position_encoding.py","file_url":"https://github.com/medicinetoken/medical-sam2/blob/HEAD/sam2_train/modeling/position_encoding.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7ceaad42da42edd5"}},{"code_sha256_prefix":"a6c3974188a5ffb0","entry":"get_1d_sine_pe","repo":"medicinetoken/medical-sam2","repo_kind":"official","path":"sam2_train/modeling/sam2_utils.py","file_url":"https://github.com/medicinetoken/medical-sam2/blob/HEAD/sam2_train/modeling/sam2_utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a6c3974188a5ffb0"}},{"code_sha256_prefix":"bb060d29525f90eb","entry":"init_t_xy","repo":"medicinetoken/medical-sam2","repo_kind":"official","path":"sam2_train/modeling/position_encoding.py","file_url":"https://github.com/medicinetoken/medical-sam2/blob/HEAD/sam2_train/modeling/position_encoding.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bb060d29525f90eb"}},{"code_sha256_prefix":"f8f09f78b474ee7c","entry":"reshape_for_broadcast","repo":"medicinetoken/medical-sam2","repo_kind":"official","path":"sam2_train/modeling/position_encoding.py","file_url":"https://github.com/medicinetoken/medical-sam2/blob/HEAD/sam2_train/modeling/position_encoding.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f8f09f78b474ee7c"}},{"code_sha256_prefix":"4ee20f090e58c425","entry":"select_closest_cond_frames","repo":"medicinetoken/medical-sam2","repo_kind":"official","path":"sam2_train/modeling/sam2_utils.py","file_url":"https://github.com/medicinetoken/medical-sam2/blob/HEAD/sam2_train/modeling/sam2_utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4ee20f090e58c425"}},{"code_sha256_prefix":"9b4931a54ea2c814","entry":"window_partition","repo":"medicinetoken/medical-sam2","repo_kind":"official","path":"sam2_train/modeling/backbones/utils.py","file_url":"https://github.com/medicinetoken/medical-sam2/blob/HEAD/sam2_train/modeling/backbones/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9b4931a54ea2c814"}},{"code_sha256_prefix":"4c236e8990640f4e","entry":"window_unpartition","repo":"medicinetoken/medical-sam2","repo_kind":"official","path":"sam2_train/modeling/backbones/utils.py","file_url":"https://github.com/medicinetoken/medical-sam2/blob/HEAD/sam2_train/modeling/backbones/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4c236e8990640f4e"}},{"code_sha256_prefix":"31a4fffe09fed1c5","entry":"build_sam2","repo":"medicinetoken/medical-sam2","repo_kind":"official","path":"sam2_train/build_sam.py","file_url":"https://github.com/medicinetoken/medical-sam2/blob/HEAD/sam2_train/build_sam.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"31a4fffe09fed1c5"}},{"code_sha256_prefix":"989075a0ddf071e4","entry":"build_sam2_video_predictor","repo":"medicinetoken/medical-sam2","repo_kind":"official","path":"sam2_train/build_sam.py","file_url":"https://github.com/medicinetoken/medical-sam2/blob/HEAD/sam2_train/build_sam.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"989075a0ddf071e4"}},{"code_sha256_prefix":"d3df9ada3e05fa55","entry":"do_pool","repo":"medicinetoken/medical-sam2","repo_kind":"official","path":"sam2_train/modeling/backbones/hieradet.py","file_url":"https://github.com/medicinetoken/medical-sam2/blob/HEAD/sam2_train/modeling/backbones/hieradet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d3df9ada3e05fa55"}},{"code_sha256_prefix":"f2e72d81f39d65cd","entry":"get_activation_fn","repo":"medicinetoken/medical-sam2","repo_kind":"official","path":"sam2_train/modeling/sam2_utils.py","file_url":"https://github.com/medicinetoken/medical-sam2/blob/HEAD/sam2_train/modeling/sam2_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f2e72d81f39d65cd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}