Open-Vocabulary Semantic Segmentation

Benchmarks archive 2025-07-28

1 leaderboard table shown for this task, 0 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

1 dataset whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

1 subtask in the archive's task tree.

Parent tasks archive 2025-07-28

Most implemented papers archive 2025-07-28

30 shown of 54 papers with code (95 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentation
  21 Mar 2023 3 repositories listed Syntology ran 5 of 6 samples · 1 unverified
  Open-vocabulary semantic segmentation presents the challenge of labeling each pixel within an image based on a wide range of text descriptions.
• Side Adapter Network for Open-Vocabulary Semantic Segmentation
  23 Feb 2023 3 repositories listed Syntology ran 1 of 9 samples · 8 unverified
  A side network is attached to a frozen CLIP model with two branches: one for predicting mask proposals, and the other for predicting attention bias which is applied in the CLIP model to recognize the class of masks.
• SegEarth-OV: Towards Training-Free Open-Vocabulary Segmentation for Remote Sensing Images
  2 Oct 2024 2 repositories listed Syntology ran 3 of 9 samples · 6 unverified · 9 pointer-only (licence)
  To tackle this issue, we propose a simple and general upsampler, SimFeatUp, to restore lost spatial information in deep features in a training-free style.
• Understanding Multi-Granularity for Open-Vocabulary Part Segmentation
  17 Jun 2024 2 repositories listed Syntology ran 11 of 15 samples · 4 unverified
  Open-vocabulary part segmentation (OVPS) is an emerging research area focused on segmenting fine-grained entities using diverse and previously unseen vocabularies.
• Panoptic Vision-Language Feature Fields
  11 Sep 2023 2 repositories listed
  In this paper, we propose to the best of our knowledge the first algorithm for open-vocabulary panoptic segmentation in 3D scenes.
• A Closer Look at the Explainability of Contrastive Language-Image Pre-training
  12 Apr 2023 2 repositories listed
  These phenomena conflict with conventional explainability methods based on the class attention map (CAM), where the raw model can highlight the local foreground regions using global supervision without alignment.
• A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Model
  29 Dec 2021 2 repositories listed
  However, semantic segmentation and the CLIP model perform on different visual granularity, that semantic segmentation processes on pixels while CLIP performs on images.
• ReME: A Data-Centric Framework for Training-Free Open-Vocabulary Segmentation
  26 Jun 2025 1 repository listed
  Training-free open-vocabulary semantic segmentation (OVS) aims to segment images given a set of arbitrary textual categories without costly model fine-tuning.
• Leveraging Depth and Language for Open-Vocabulary Domain-Generalized Semantic Segmentation
  11 Jun 2025 1 repository listed
  Open-Vocabulary semantic segmentation (OVSS) and domain generalization in semantic segmentation (DGSS) highlight a subtle complementarity that motivates Open-Vocabulary Domain-Generalized Semantic Segmentation (OV-DGSS).
• Test-Time Adaptation of Vision-Language Models for Open-Vocabulary Semantic Segmentation
  28 May 2025 1 repository listed
  Recently, test-time adaptation has attracted wide interest in the context of vision-language models for image classification.
• OpenSeg-R: Improving Open-Vocabulary Segmentation via Step-by-Step Visual Reasoning
  22 May 2025 1 repository listed
  To the best of our knowledge, OpenSeg-R is the first framework to introduce explicit step-by-step visual reasoning into OVS.
• FLOSS: Free Lunch in Open-vocabulary Semantic Segmentation
  14 Apr 2025 1 repository listed
  Our plug-and-play method, coined FLOSS, is orthogonal and complementary to existing OVSS methods, offering a ''free lunch'' to systematically improve OVSS without labels and additional training.
• Semantic Library Adaptation: LoRA Retrieval and Fusion for Open-Vocabulary Semantic Segmentation
  27 Mar 2025 1 repository listed
  Open-vocabulary semantic segmentation models associate vision and text to label pixels from an undefined set of classes using textual queries, providing versatile performance on novel datasets.
• LPOSS: Label Propagation Over Patches and Pixels for Open-vocabulary Semantic Segmentation
  25 Mar 2025 1 repository listed Syntology ran 0 of 5 samples · 5 unverified
  We propose a training-free method for open-vocabulary semantic segmentation using Vision-and-Language Models (VLMs).
• Efficient Redundancy Reduction for Open-Vocabulary Semantic Segmentation
  29 Jan 2025 1 repository listed
  Open-vocabulary semantic segmentation (OVSS) is an open-world task that aims to assign each pixel within an image to a specific class defined by arbitrary text descriptions.
• Fine-Grained Image-Text Correspondence with Cost Aggregation for Open-Vocabulary Part Segmentation
  16 Jan 2025 1 repository listed Syntology ran 7 of 11 samples · 4 unverified
  Open-Vocabulary Part Segmentation (OVPS) is an emerging field for recognizing fine-grained parts in unseen categories.
• FGAseg: Fine-Grained Pixel-Text Alignment for Open-Vocabulary Semantic Segmentation
  1 Jan 2025 1 repository listed
  The core of FGAseg is a Pixel-Level Alignment module that employs a cross-modal attention mechanism and a text-pixel alignment loss to refine the coarse-grained alignment from CLIP, achieving finer-grained pixel-text…
• DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignment
  20 Dec 2024 1 repository listed
  Self-supervised visual foundation models produce powerful embeddings that achieve remarkable performance on a wide range of downstream tasks.
• MaskCLIP++: A Mask-Based CLIP Fine-tuning Framework for Open-Vocabulary Image Segmentation
  16 Dec 2024 1 repository listed
  Open-vocabulary image segmentation has been advanced through the synergy between mask generators and vision-language models like Contrastive Language-Image Pre-training (CLIP).
• Mask-Adapter: The Devil is in the Masks for Open-Vocabulary Segmentation
  5 Dec 2024 1 repository listed
  Mask-Adapter integrates seamlessly into open-vocabulary segmentation methods based on mask pooling in a plug-and-play manner, delivering more accurate classification results.
• CLIPer: Hierarchically Improving Spatial Representation of CLIP for Open-Vocabulary Semantic Segmentation
  21 Nov 2024 1 repository listed
  The proposed CLIPer includes an early-layer fusion module and a fine-grained compensation module.
• XMask3D: Cross-modal Mask Reasoning for Open Vocabulary 3D Semantic Segmentation
  20 Nov 2024 1 repository listed Syntology ran 0 of 11 samples · 11 unverified
  In our approach, we developed a mask generator based on the denoising UNet from a pre-trained diffusion model, leveraging its capability for precise textual control over dense pixel representations and enhancing the…
• ITACLIP: Boosting Training-Free Semantic Segmentation with Image, Text, and Architectural Enhancements
  18 Nov 2024 1 repository listed
  Recent advances in foundational Vision Language Models (VLMs) have reshaped the evaluation paradigm in computer vision tasks.
• CorrCLIP: Reconstructing Correlations in CLIP with Off-the-Shelf Foundation Models for Open-Vocabulary Semantic Segmentation
  15 Nov 2024 1 repository listed Syntology ran 0 of 9 samples · 9 unverified · 9 pointer-only (licence)
  Open-vocabulary semantic segmentation aims to assign semantic labels to each pixel without relying on a predefined set of categories.
• OVOSE: Open-Vocabulary Semantic Segmentation in Event-Based Cameras
  18 Aug 2024 1 repository listed
  In this paper, we introduce OVOSE, the first Open-Vocabulary Semantic Segmentation algorithm for Event cameras.
• In Defense of Lazy Visual Grounding for Open-Vocabulary Semantic Segmentation
  9 Aug 2024 1 repository listed
  We present lazy visual grounding, a two-stage approach of unsupervised object mask discovery followed by object grounding, for open-vocabulary semantic segmentation.
• ProxyCLIP: Proxy Attention Improves CLIP for Open-Vocabulary Segmentation
  9 Aug 2024 1 repository listed Syntology ran 3 of 9 samples · 6 unverified · 9 pointer-only (licence)
  ProxyCLIP leverages the spatial feature correspondence from VFMs as a form of proxy attention to augment CLIP, thereby inheriting the VFMs' robust local consistency and maintaining CLIP's exceptional zero-shot transfer…
• Explore the Potential of CLIP for Training-Free Open Vocabulary Semantic Segmentation
  11 Jul 2024 1 repository listed Syntology ran 3 of 5 samples · 2 unverified
  CLIP, as a vision-language model, has significantly advanced Open-Vocabulary Semantic Segmentation (OVSS) with its zero-shot capabilities.
• A Unified Framework for 3D Scene Understanding
  3 Jul 2024 1 repository listed Syntology ran 7 of 7 samples · 0 unverified
  We propose UniSeg3D, a unified 3D scene understanding framework that achieves panoptic, semantic, instance, interactive, referring, and open-vocabulary segmentation tasks within a single model.
• Open-Vocabulary Semantic Segmentation with Image Embedding Balancing
  14 Jun 2024 1 repository listed Syntology ran 3 of 12 samples · 9 unverified
  To learn a consistent semantic structure from CLIP, the SSC Loss aligns the inter-classes affinity in the image feature space with that in the text feature space of CLIP, thereby improving the generalization ability of…

Syntology lines on 12 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.