{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/simpleclick-interactive-image-segmentation","title":"SimpleClick: Interactive Image Segmentation with Simple Vision Transformers","arxiv_id":"2210.11006","date":"2022-10-20","proceeding":"ICCV 2023 1","authors":["Qin Liu","Zhenlin Xu","Gedas Bertasius","Marc Niethammer"],"abstract":"Click-based interactive image segmentation aims at extracting objects with a limited user clicking. A hierarchical backbone is the de-facto architecture for current methods. Recently, the plain, non-hierarchical Vision Transformer (ViT) has emerged as a competitive backbone for dense prediction tasks. This design allows the original ViT to be a foundation model that can be finetuned for downstream tasks without redesigning a hierarchical backbone for pretraining. Although this design is simple and has been proven effective, it has not yet been explored for interactive image segmentation. To fill this gap, we propose SimpleClick, the first interactive segmentation method that leverages a plain backbone. Based on the plain backbone, we introduce a symmetric patch embedding layer that encodes clicks into the backbone with minor modifications to the backbone itself. With the plain backbone pretrained as a masked autoencoder (MAE), SimpleClick achieves state-of-the-art performance. Remarkably, our method achieves 4.15 NoC@90 on SBD, improving 21.8% over the previous best result. Extensive evaluation on medical images demonstrates the generalizability of our method. We further develop an extremely tiny ViT backbone for SimpleClick and provide a detailed computational analysis, highlighting its suitability as a practical annotation tool.","url_abs":"https://arxiv.org/abs/2210.11006v3","url_pdf":"https://arxiv.org/pdf/2210.11006v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"simpleclick-interactive-image-segmentation","repo_url":"https://github.com/uncbiag/simpleclick","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"simpleclick-interactive-image-segmentation","repo_url":"https://github.com/yihanhu-2022/diffmatte","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"image-segmentation","task_name":"Image Segmentation"},{"task_slug":"interactive-segmentation","task_name":"Interactive Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"},{"method_slug":"vision-transformer","method_name":"Vision Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/interactive-segmentation-on-berkeley","task":"Interactive Segmentation","dataset":"Berkeley","model":"SimpleClick (ViT-H, C+L)","rank_in_archive_order":4,"of":14,"metrics":{"NoC@90":"1.75"},"uses_additional_data":false},{"leaderboard":"/sota/interactive-segmentation-on-berkeley","task":"Interactive Segmentation","dataset":"Berkeley","model":"SimpleClick (ViT-H, SBD)","rank_in_archive_order":5,"of":14,"metrics":{"NoC@90":"2.09"},"uses_additional_data":false},{"leaderboard":"/sota/interactive-segmentation-on-davis","task":"Interactive Segmentation","dataset":"DAVIS","model":"SimpleClick (ViT-H, C+L)","rank_in_archive_order":4,"of":15,"metrics":{"NoC@85":"3.41","NoC@90":"4.70"},"uses_additional_data":false},{"leaderboard":"/sota/interactive-segmentation-on-davis","task":"Interactive Segmentation","dataset":"DAVIS","model":"SimpleClick (ViT-H, SBD)","rank_in_archive_order":7,"of":15,"metrics":{"NoC@85":"4.20","NoC@90":"5.34"},"uses_additional_data":false},{"leaderboard":"/sota/interactive-segmentation-on-grabcut","task":"Interactive Segmentation","dataset":"GrabCut","model":"SimpleClick (ViT-L, C+L)","rank_in_archive_order":2,"of":18,"metrics":{"NoC@85":"1.32","NoC@90":"1.40"},"uses_additional_data":false},{"leaderboard":"/sota/interactive-segmentation-on-grabcut","task":"Interactive Segmentation","dataset":"GrabCut","model":"SimpleClick (ViT-H, SBD)","rank_in_archive_order":4,"of":18,"metrics":{"NoC@85":"1.32","NoC@90":"1.44"},"uses_additional_data":false},{"leaderboard":"/sota/interactive-segmentation-on-sbd","task":"Interactive Segmentation","dataset":"SBD","model":"SimpleClick","rank_in_archive_order":2,"of":14,"metrics":{"NoC@85":"2.51","NoC@90":"4.15"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2210.11006","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.11006"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/uncbiag/simpleclick","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yihanhu-2022/diffmatte","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":1,"ran":1,"unverified":1},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"fca7b0a20cde7062","entry":"get_predictor_and_zoomin_params","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"fca7b0a20cde7062"}},{"code_sha256_prefix":"954d2116dbc85a7b","entry":"handle_exception","repo":"uncbiag/simpleclick","repo_kind":"official","path":"interactive_demo/canvas.py","file_url":"https://github.com/uncbiag/simpleclick/blob/HEAD/interactive_demo/canvas.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"954d2116dbc85a7b"}},{"code_sha256_prefix":"8bb23b71c17a0a25","entry":"split_points_by_order","repo":"uncbiag/simpleclick","repo_kind":"official","path":"isegm/model/is_model.py","file_url":"https://github.com/uncbiag/simpleclick/blob/HEAD/isegm/model/is_model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8bb23b71c17a0a25"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}