Open Vocabulary Semantic Segmentation

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

15 leaderboard tables shown for this task, 14 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted. 10 shown of 15 until expanded.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

6 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

1 subtask in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 71 papers with code (113 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement
  9 Mar 2025 3 repositories listed
  Traditional methods for reasoning segmentation rely on supervised fine-tuning with categorical labels and simple descriptions, limiting its out-of-domain generalization and lacking explicit reasoning processes.
• CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentation
  21 Mar 2023 3 repositories listed Syntology ran 5 of 6 samples · 1 unverified
  Open-vocabulary semantic segmentation presents the challenge of labeling each pixel within an image based on a wide range of text descriptions.
• Side Adapter Network for Open-Vocabulary Semantic Segmentation
  23 Feb 2023 3 repositories listed Syntology ran 1 of 9 samples · 8 unverified
  A side network is attached to a frozen CLIP model with two branches: one for predicting mask proposals, and the other for predicting attention bias which is applied in the CLIP model to recognize the class of masks.
• SegEarth-OV: Towards Training-Free Open-Vocabulary Segmentation for Remote Sensing Images
  2 Oct 2024 2 repositories listed Syntology ran 3 of 9 samples · 6 unverified · 9 pointer-only (licence)
  To tackle this issue, we propose a simple and general upsampler, SimFeatUp, to restore lost spatial information in deep features in a training-free style.
• Understanding Multi-Granularity for Open-Vocabulary Part Segmentation
  17 Jun 2024 2 repositories listed Syntology ran 11 of 15 samples · 4 unverified
  Open-vocabulary part segmentation (OVPS) is an emerging research area focused on segmenting fine-grained entities using diverse and previously unseen vocabularies.
• Open-Vocabulary Segmentation with Semantic-Assisted Calibration
  7 Dec 2023 2 repositories listed Syntology ran 5 of 10 samples · 5 unverified · 10 pointer-only (licence)
  We attribute this to the in-vocabulary embedding and domain-biased CLIP prediction.
• Learning Mask-aware CLIP Representations for Zero-Shot Segmentation
  30 Sep 2023 2 repositories listed Syntology ran 1 of 7 samples · 6 unverified · 2 pointer-only (licence)
  However, in the paper, we reveal that CLIP is insensitive to different mask proposals and tends to produce similar predictions for various mask proposals of the same image.
• Panoptic Vision-Language Feature Fields
  11 Sep 2023 2 repositories listed
  In this paper, we propose to the best of our knowledge the first algorithm for open-vocabulary panoptic segmentation in 3D scenes.
• A Closer Look at the Explainability of Contrastive Language-Image Pre-training
  12 Apr 2023 2 repositories listed
  These phenomena conflict with conventional explainability methods based on the class attention map (CAM), where the raw model can highlight the local foreground regions using global supervision without alignment.
• A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Model
  29 Dec 2021 2 repositories listed
  However, semantic segmentation and the CLIP model perform on different visual granularity, that semantic segmentation processes on pixels while CLIP performs on images.
• ReME: A Data-Centric Framework for Training-Free Open-Vocabulary Segmentation
  26 Jun 2025 1 repository listed
  Training-free open-vocabulary semantic segmentation (OVS) aims to segment images given a set of arbitrary textual categories without costly model fine-tuning.
• Leveraging Depth and Language for Open-Vocabulary Domain-Generalized Semantic Segmentation
  11 Jun 2025 1 repository listed
  Open-Vocabulary semantic segmentation (OVSS) and domain generalization in semantic segmentation (DGSS) highlight a subtle complementarity that motivates Open-Vocabulary Domain-Generalized Semantic Segmentation (OV-DGSS).
• Test-Time Adaptation of Vision-Language Models for Open-Vocabulary Semantic Segmentation
  28 May 2025 1 repository listed
  Recently, test-time adaptation has attracted wide interest in the context of vision-language models for image classification.
• OpenSeg-R: Improving Open-Vocabulary Segmentation via Step-by-Step Visual Reasoning
  22 May 2025 1 repository listed
  To the best of our knowledge, OpenSeg-R is the first framework to introduce explicit step-by-step visual reasoning into OVS.
• Show or Tell? A Benchmark To Evaluate Visual and Textual Prompts in Semantic Segmentation
  6 May 2025 1 repository listed
  Prompt engineering has shown remarkable success with large language models, yet its systematic exploration in computer vision remains limited.
• FLOSS: Free Lunch in Open-vocabulary Semantic Segmentation
  14 Apr 2025 1 repository listed
  Our plug-and-play method, coined FLOSS, is orthogonal and complementary to existing OVSS methods, offering a ''free lunch'' to systematically improve OVSS without labels and additional training.
• Semantic Library Adaptation: LoRA Retrieval and Fusion for Open-Vocabulary Semantic Segmentation
  27 Mar 2025 1 repository listed
  Open-vocabulary semantic segmentation models associate vision and text to label pixels from an undefined set of classes using textual queries, providing versatile performance on novel datasets.
• LPOSS: Label Propagation Over Patches and Pixels for Open-vocabulary Semantic Segmentation
  25 Mar 2025 1 repository listed Syntology ran 0 of 5 samples · 5 unverified
  We propose a training-free method for open-vocabulary semantic segmentation using Vision-and-Language Models (VLMs).
• Efficient Redundancy Reduction for Open-Vocabulary Semantic Segmentation
  29 Jan 2025 1 repository listed
  Open-vocabulary semantic segmentation (OVSS) is an open-world task that aims to assign each pixel within an image to a specific class defined by arbitrary text descriptions.
• Fine-Grained Image-Text Correspondence with Cost Aggregation for Open-Vocabulary Part Segmentation
  16 Jan 2025 1 repository listed Syntology ran 7 of 11 samples · 4 unverified
  Open-Vocabulary Part Segmentation (OVPS) is an emerging field for recognizing fine-grained parts in unseen categories.
• FGAseg: Fine-Grained Pixel-Text Alignment for Open-Vocabulary Semantic Segmentation
  1 Jan 2025 1 repository listed
  The core of FGAseg is a Pixel-Level Alignment module that employs a cross-modal attention mechanism and a text-pixel alignment loss to refine the coarse-grained alignment from CLIP, achieving finer-grained pixel-text…
• DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignment
  20 Dec 2024 1 repository listed
  Self-supervised visual foundation models produce powerful embeddings that achieve remarkable performance on a wide range of downstream tasks.
• MaskCLIP++: A Mask-Based CLIP Fine-tuning Framework for Open-Vocabulary Image Segmentation
  16 Dec 2024 1 repository listed
  Open-vocabulary image segmentation has been advanced through the synergy between mask generators and vision-language models like Contrastive Language-Image Pre-training (CLIP).
• Mask-Adapter: The Devil is in the Masks for Open-Vocabulary Segmentation
  5 Dec 2024 1 repository listed
  Mask-Adapter integrates seamlessly into open-vocabulary segmentation methods based on mask pooling in a plug-and-play manner, delivering more accurate classification results.
• HyperSeg: Towards Universal Visual Segmentation with Large Language Model
  26 Nov 2024 1 repository listed Syntology ran 7 of 17 samples · 10 unverified
  This paper aims to address universal segmentation for image and video perception with the strong reasoning ability empowered by Visual Large Language Models (VLLMs).
• CLIPer: Hierarchically Improving Spatial Representation of CLIP for Open-Vocabulary Semantic Segmentation
  21 Nov 2024 1 repository listed
  The proposed CLIPer includes an early-layer fusion module and a fine-grained compensation module.
• XMask3D: Cross-modal Mask Reasoning for Open Vocabulary 3D Semantic Segmentation
  20 Nov 2024 1 repository listed Syntology ran 0 of 11 samples · 11 unverified
  In our approach, we developed a mask generator based on the denoising UNet from a pre-trained diffusion model, leveraging its capability for precise textual control over dense pixel representations and enhancing the…
• ITACLIP: Boosting Training-Free Semantic Segmentation with Image, Text, and Architectural Enhancements
  18 Nov 2024 1 repository listed
  Recent advances in foundational Vision Language Models (VLMs) have reshaped the evaluation paradigm in computer vision tasks.
• CorrCLIP: Reconstructing Correlations in CLIP with Off-the-Shelf Foundation Models for Open-Vocabulary Semantic Segmentation
  15 Nov 2024 1 repository listed Syntology ran 0 of 9 samples · 9 unverified · 9 pointer-only (licence)
  Open-vocabulary semantic segmentation aims to assign semantic labels to each pixel without relying on a predefined set of categories.
• OVOSE: Open-Vocabulary Semantic Segmentation in Event-Based Cameras
  18 Aug 2024 1 repository listed
  In this paper, we introduce OVOSE, the first Open-Vocabulary Semantic Segmentation algorithm for Event cameras.

Syntology lines on 11 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.