{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/nninteractive-redefining-3d-promptable","title":"nnInteractive: Redefining 3D Promptable Segmentation","arxiv_id":"2503.08373","date":"2025-03-11","proceeding":null,"authors":["Fabian Isensee","Maximilian Rokuss","Lars Krämer","Stefan Dinkelacker","Ashis Ravindran","Florian Stritzke","Benjamin Hamm","Tassilo Wald","Moritz Langenberg","Constantin Ulrich","Jonathan Deissler","Ralf Floca","Klaus Maier-Hein"],"abstract":"Accurate and efficient 3D segmentation is essential for both clinical and research applications. While foundation models like SAM have revolutionized interactive segmentation, their 2D design and domain shift limitations make them ill-suited for 3D medical images. Current adaptations address some of these challenges but remain limited, either lacking volumetric awareness, offering restricted interactivity, or supporting only a small set of structures and modalities. Usability also remains a challenge, as current tools are rarely integrated into established imaging platforms and often rely on cumbersome web-based interfaces with restricted functionality. We introduce nnInteractive, the first comprehensive 3D interactive open-set segmentation method. It supports diverse prompts-including points, scribbles, boxes, and a novel lasso prompt-while leveraging intuitive 2D interactions to generate full 3D segmentations. Trained on 120+ diverse volumetric 3D datasets (CT, MRI, PET, 3D Microscopy, etc.), nnInteractive sets a new state-of-the-art in accuracy, adaptability, and usability. Crucially, it is the first method integrated into widely used image viewers (e.g., Napari, MITK), ensuring broad accessibility for real-world clinical and research applications. Extensive benchmarking demonstrates that nnInteractive far surpasses existing methods, setting a new standard for AI-driven interactive 3D segmentation. nnInteractive is publicly available: https://github.com/MIC-DKFZ/napari-nninteractive (Napari plugin), https://www.mitk.org/MITK-nnInteractive (MITK integration), https://github.com/MIC-DKFZ/nnInteractive (Python backend).","url_abs":"https://arxiv.org/abs/2503.08373v1","url_pdf":"https://arxiv.org/pdf/2503.08373v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"nninteractive-redefining-3d-promptable","repo_url":"https://github.com/mic-dkfz/napari-nninteractive","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"nninteractive-redefining-3d-promptable","repo_url":"https://github.com/mic-dkfz/nninteractive","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"nninteractive-redefining-3d-promptable","repo_url":"https://github.com/coendevente/nninteractive-slicer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"nninteractive-redefining-3d-promptable","repo_url":"https://github.com/mic-dkfz/lesionlocator","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"GPL-3.0"}}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"interactive-segmentation","task_name":"Interactive Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"}],"methods":[{"method_slug":"sam","method_name":"SAM"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2503.08373","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.08373"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mic-dkfz/nninteractive","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mic-dkfz/napari-nninteractive","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/coendevente/nninteractive-slicer","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mic-dkfz/lesionlocator","reach":{"status":"ok","spdx":"GPL-3.0"}}],"summary":{"ran_honours":2,"ran_fixture":1,"unverified":6},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":2},"listed":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1753045630de2952","entry":"infer_num_interaction_channels_from_mapping","repo":"mic-dkfz/nninteractive","repo_kind":"official","path":"nnInteractive/utils/inference_helpers.py","file_url":"https://github.com/mic-dkfz/nninteractive/blob/HEAD/nnInteractive/utils/inference_helpers.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1753045630de2952"}},{"code_sha256_prefix":"35893934de5bfe1f","entry":"parse_channel_pair","repo":"mic-dkfz/nninteractive","repo_kind":"official","path":"nnInteractive/utils/inference_helpers.py","file_url":"https://github.com/mic-dkfz/nninteractive/blob/HEAD/nnInteractive/utils/inference_helpers.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"35893934de5bfe1f"}},{"code_sha256_prefix":"7c31678ef62901a0","entry":"version_to_tuple","repo":"mic-dkfz/nninteractive","repo_kind":"official","path":"nnInteractive/utils/inference_helpers.py","file_url":"https://github.com/mic-dkfz/nninteractive/blob/HEAD/nnInteractive/utils/inference_helpers.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7c31678ef62901a0"}},{"code_sha256_prefix":"86ce0a7887fb9dc4","entry":"bbox","repo":"coendevente/nninteractive-slicer","repo_kind":"listed","path":"slicer_plugin/SlicerNNInteractive/Testing/Python/SlicerNNInteractiveSegmentationTest.py","file_url":"https://github.com/coendevente/nninteractive-slicer/blob/HEAD/slicer_plugin/SlicerNNInteractive/Testing/Python/SlicerNNInteractiveSegmentationTest.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"86ce0a7887fb9dc4"}},{"code_sha256_prefix":"67adb1cb306d8b9d","entry":"ensure_synched","repo":"coendevente/nninteractive-slicer","repo_kind":"listed","path":"slicer_plugin/SlicerNNInteractive/SlicerNNInteractive.py","file_url":"https://github.com/coendevente/nninteractive-slicer/blob/HEAD/slicer_plugin/SlicerNNInteractive/SlicerNNInteractive.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"67adb1cb306d8b9d"}},{"code_sha256_prefix":"af5dbe64a8f484b8","entry":"left_button_only","repo":"mic-dkfz/napari-nninteractive","repo_kind":"official","path":"src/napari_nninteractive/mouse_bindings.py","file_url":"https://github.com/mic-dkfz/napari-nninteractive/blob/HEAD/src/napari_nninteractive/mouse_bindings.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"af5dbe64a8f484b8"}},{"code_sha256_prefix":"0e039536283a75f5","entry":"negative","repo":"coendevente/nninteractive-slicer","repo_kind":"listed","path":"slicer_plugin/SlicerNNInteractive/Testing/Python/SlicerNNInteractiveSegmentationTest.py","file_url":"https://github.com/coendevente/nninteractive-slicer/blob/HEAD/slicer_plugin/SlicerNNInteractive/Testing/Python/SlicerNNInteractiveSegmentationTest.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0e039536283a75f5"}},{"code_sha256_prefix":"d844a7a70ed449b0","entry":"positive","repo":"coendevente/nninteractive-slicer","repo_kind":"listed","path":"slicer_plugin/SlicerNNInteractive/Testing/Python/SlicerNNInteractiveSegmentationTest.py","file_url":"https://github.com/coendevente/nninteractive-slicer/blob/HEAD/slicer_plugin/SlicerNNInteractive/Testing/Python/SlicerNNInteractiveSegmentationTest.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d844a7a70ed449b0"}},{"code_sha256_prefix":"7f5eaa04a2e1d50d","entry":"torch_compile_unsupported_reason","repo":"coendevente/nninteractive-slicer","repo_kind":"listed","path":"slicer_plugin/SlicerNNInteractive/SlicerNNInteractive.py","file_url":"https://github.com/coendevente/nninteractive-slicer/blob/HEAD/slicer_plugin/SlicerNNInteractive/SlicerNNInteractive.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7f5eaa04a2e1d50d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}