</> Code with Papers by Syntology

Papers › MaskCLIP++: A Mask-Based CLIP Fine-tuning Framework for Open-Vocabulary Image Segmentation

MaskCLIP++: A Mask-Based CLIP Fine-tuning Framework for Open-Vocabulary Image Segmentation

16 Dec 2024arXiv:2412.11464archive 2025-07-28

Quan-Sheng Zeng, Yunheng Li, Daquan Zhou, Guanbin Li, Qibin Hou, Ming-Ming Cheng

Open-vocabulary image segmentation has been advanced through the synergy between mask generators and vision-language models like Contrastive Language-Image Pre-training (CLIP). Previous approaches focus on generating masks while aligning mask features with text embeddings during training. In this paper, we observe that relying on generated low-quality masks can weaken the alignment of vision and language in regional representations. This motivates us to present a new fine-tuning framework, named MaskCLIP++, which uses ground-truth masks instead of generated masks to enhance the mask classification capability of CLIP. Due to the limited diversity of image segmentation datasets with mask annotations, we propose incorporating a consistency alignment constraint during fine-tuning, which alleviates categorical bias toward the fine-tuning dataset. After low-cost fine-tuning, combining with the mask generator in previous state-of-the-art mask-based open vocabulary segmentation methods, we achieve performance improvements of +1.7, +2.3, +2.1, +3.1, and +0.3 mIoU on the A-847, PC-459, A-150, PC-59, and PAS-20 datasets, respectively. Code is released at https://github.com/HVision-NKU/MaskCLIPpp .

PaperPDFCode

Code

hvision-nku/maskclippp officialmentioned in papermentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Image SegmentationOpen Vocabulary Semantic SegmentationOpen-Vocabulary Semantic SegmentationSegmentationSemantic Segmentation

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Open Vocabulary Semantic Segmentation ADE20K-150 MaskCLIP++ mIoU 38.2 #2 of 23 Archive leaderboard report
Open Vocabulary Semantic Segmentation ADE20K-847 MaskCLIP++ mIoU 16.8 #2 of 19 Archive leaderboard report
Open Vocabulary Semantic Segmentation PASCAL Context-459 MaskCLIP++ mIoU 23.9 #3 of 15 Archive leaderboard report
Open Vocabulary Semantic Segmentation PASCAL Context-59 MaskCLIP++ mIoU 62.5 #4 of 24 Archive leaderboard report
Open Vocabulary Semantic Segmentation PascalVOC-20 MaskCLIP++ mIoU 96.8 #5 of 20 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

CLIPFocus

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections