Zero Shot Segmentation

Benchmarks archive 2025-07-28

2 leaderboard tables shown for this task, 2 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

3 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 70 papers with code (134 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection
  9 Mar 2023 10 repositories listed Syntology ran 2 of 5 samples · 3 unverified
  To effectively fuse language and vision modalities, we conceptually divide a closed-set detector into three phases and propose a tight fusion solution, which includes a feature enhancer, a language-guided query…
• Image Segmentation Using Text and Image Prompts
  18 Dec 2021 6 repositories listed Syntology ran 1 of 2 samples · 1 unverified · 2 pointer-only (licence)
  After training on an extended version of the PhraseCut dataset, our system generates a binary segmentation map for an image based on a free-text prompt or on an additional image expressing the query.
• Segment Anything in High Quality
  2 Jun 2023 4 repositories listed Syntology ran 3 of 17 samples · 14 unverified · 3 pointer-only (licence)
  HQ-SAM is only trained on the introduced detaset of 44k masks, which takes only 4 hours on 8 GPUs.
• Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement
  9 Mar 2025 3 repositories listed
  Traditional methods for reasoning segmentation rely on supervised fine-tuning with categorical labels and simple descriptions, limiting its out-of-domain generalization and lacking explicit reasoning processes.
• One-Prompt to Segment All Medical Images
  17 May 2023 3 repositories listed Syntology ran 3 of 4 samples · 1 unverified
  Tested on 14 previously unseen datasets, the One-Prompt Model showcases superior zero-shot segmentation capabilities, outperforming a wide range of related methods.
• Side Adapter Network for Open-Vocabulary Semantic Segmentation
  23 Feb 2023 3 repositories listed Syntology ran 1 of 9 samples · 8 unverified
  A side network is attached to a frozen CLIP model with two branches: one for predicting mask proposals, and the other for predicting attention bias which is applied in the CLIP model to recognize the class of masks.
• MedCLIP-SAMv2: Towards Universal Text-Driven Medical Image Segmentation
  28 Sep 2024 2 repositories listed
  Recently, the introduction of foundation models like CLIP and Segment-Anything-Model (SAM), with robust cross-domain representations, has paved the way for interactive and universal image segmentation.
• Learning Mask-aware CLIP Representations for Zero-Shot Segmentation
  30 Sep 2023 2 repositories listed Syntology ran 1 of 7 samples · 6 unverified · 2 pointer-only (licence)
  However, in the paper, we reveal that CLIP is insensitive to different mask proposals and tends to produce similar predictions for various mask proposals of the same image.
• Segment Anything Model for Medical Image Analysis: an Experimental Study
  20 Apr 2023 2 repositories listed
  We conclude that SAM shows impressive zero-shot segmentation performance for certain medical imaging datasets, but moderate to poor performance for others.
• A Closer Look at the Explainability of Contrastive Language-Image Pre-training
  12 Apr 2023 2 repositories listed
  These phenomena conflict with conventional explainability methods based on the class attention map (CAM), where the raw model can highlight the local foreground regions using global supervision without alignment.
• A Simple Framework for Open-Vocabulary Segmentation and Detection
  14 Mar 2023 2 repositories listed
  We present OpenSeeD, a simple Open-vocabulary Segmentation and Detection framework that jointly learns from different segmentation and detection datasets.
• Context-aware Feature Generation for Zero-shot Semantic Segmentation
  16 Aug 2020 2 repositories listed
  In this paper, we propose a novel context-aware feature generation method for zero-shot segmentation named CaGNet.
• Compress Any Segment Anything Model (SAM)
  11 Jul 2025 1 repository listed
  Due to the excellent performance in yielding high-quality, zero-shot segmentation, Segment Anything Model (SAM) and its variants have been widely applied in diverse scenarios such as healthcare and intelligent…
• Zero-Shot Tree Detection and Segmentation from Aerial Forest Imagery
  3 Jun 2025 1 repository listed
  In this paper, we investigate the efficacy of using a state-of-the-art image segmentation model, Segment Anything Model 2 (SAM2), in a zero-shot manner for individual tree detection and segmentation.
• Removing Watermarks with Partial Regeneration using Semantic Information
  13 May 2025 1 repository listed
  As AI-generated imagery becomes ubiquitous, invisible watermarks have emerged as a primary line of defense for copyright and provenance.
• 3D-PointZshotS: Geometry-Aware 3D Point Cloud Zero-Shot Semantic Segmentation Narrowing the Visual-Semantic Gap
  16 Apr 2025 1 repository listed
  Existing zero-shot 3D point cloud segmentation methods often struggle with limited transferability from seen classes to unseen classes and from semantic to visual space.
• CellViT++: Energy-Efficient and Adaptive Cell Segmentation and Classification Using Foundation Models
  9 Jan 2025 1 repository listed
  Digital Pathology is a cornerstone in the diagnosis and treatment of diseases.
• HOLa: HoloLens Object Labeling
  6 Dec 2024 1 repository listed
  In the context of medical Augmented Reality (AR) applications, object tracking is a key challenge and requires a significant amount of annotation masks.
• Quantifying the Limits of Segmentation Foundation Models: Modeling Challenges in Segmenting Tree-Like and Low-Contrast Objects
  5 Dec 2024 1 repository listed
  Image segmentation foundation models (SFMs) like Segment Anything Model (SAM) have achieved impressive zero-shot and interactive segmentation across diverse domains.
• COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training
  2 Dec 2024 1 repository listed
  Vision-Language Models (VLMs) trained with contrastive loss have achieved significant advancements in various vision and language tasks.
• 3DGS-CD: 3D Gaussian Splatting-based Change Detection for Physical Object Rearrangement
  6 Nov 2024 1 repository listed
  We present 3DGS-CD, the first 3D Gaussian Splatting (3DGS)-based method for detecting physical object rearrangements in 3D scenes.
• ZIM: Zero-Shot Image Matting for Anything
  1 Nov 2024 1 repository listed
  To address this limitation, we propose a novel zero-shot image matting model, called ZIM, with two key contributions: First, we develop a label converter that transforms segmentation labels into detailed matte labels,…
• Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations
  3 Oct 2024 1 repository listed Syntology ran 2 of 3 samples · 1 unverified
  We investigate the internal representations of vision-language models (VLMs) to address hallucinations, a persistent challenge despite advances in model size and training.
• Evaluation Study on SAM 2 for Class-agnostic Instance-level Segmentation
  4 Sep 2024 1 repository listed
  Segment Anything Model (SAM) has demonstrated powerful zero-shot segmentation performance in natural scenes.
• Unleashing the Potential of SAM2 for Biomedical Images and Videos: A Survey
  23 Aug 2024 1 repository listed
  The unprecedented developments in segmentation foundational models have become a dominant force in the field of computer vision, introducing a multitude of previously unexplored capabilities in a wide range of natural…
• SAM-UNet:Enhancing Zero-Shot Segmentation of SAM for Universal Medical Images
  19 Aug 2024 1 repository listed
  We train SAM-UNet on SA-Med2D-16M, the largest 2-dimensional medical image segmentation dataset to date, yielding a universal pretrained model for medical images.
• PaveCap: The First Multimodal Framework for Comprehensive Pavement Condition Assessment with Dense Captioning and PCI Estimation
  7 Aug 2024 1 repository listed
  The framework consists of two main parts: a Single-Shot PCI Estimation Network and a Dense Captioning Network.
• Zero-Shot Surgical Tool Segmentation in Monocular Video Using Segment Anything Model 2
  3 Aug 2024 1 repository listed
  The Segment Anything Model 2 (SAM 2) is the latest generation foundation model for image and video segmentation.
• X-Recon: Learning-based Patient-specific High-Resolution CT Reconstruction from Orthogonal X-Ray Images
  22 Jul 2024 1 repository listed
  Rapid and accurate diagnosis of pneumothorax, utilizing chest X-ray and computed tomography (CT), is crucial for assisted diagnosis.
• MeshSegmenter: Zero-Shot Mesh Semantic Segmentation via Texture Synthesis
  18 Jul 2024 1 repository listed
  Specifically, our model leverages the Segment Anything Model (SAM) model to segment the target regions from images rendered from the 3D shape.

Syntology lines on 7 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.