</> Code with Papers by Syntology

Papers › SemiVL: Semi-Supervised Semantic Segmentation with Vision-Language Guidance

SemiVL: Semi-Supervised Semantic Segmentation with Vision-Language Guidance

27 Nov 2023arXiv:2311.16241archive 2025-07-28

Lukas Hoyer, David Joseph Tan, Muhammad Ferjad Naeem, Luc van Gool, Federico Tombari

In semi-supervised semantic segmentation, a model is trained with a limited number of labeled images along with a large corpus of unlabeled images to reduce the high annotation effort. While previous methods are able to learn good segmentation boundaries, they are prone to confuse classes with similar visual appearance due to the limited supervision. On the other hand, vision-language models (VLMs) are able to learn diverse semantic knowledge from image-caption datasets but produce noisy segmentation due to the image-level training. In SemiVL, we propose to integrate rich priors from VLM pre-training into semi-supervised semantic segmentation to learn better semantic decision boundaries. To adapt the VLM from global to local reasoning, we introduce a spatial fine-tuning strategy for label-efficient learning. Further, we design a language-guided decoder to jointly reason over vision and language. Finally, we propose to handle inherent ambiguities in class labels by providing the model with language guidance in the form of class definitions. We evaluate SemiVL on 4 semantic segmentation datasets, where it significantly outperforms previous semi-supervised methods. For instance, SemiVL improves the state-of-the-art by +13.5 mIoU on COCO with 232 annotated images and by +6.1 mIoU on Pascal VOC with 92 labels. Project page: https://github.com/google-research/semivl

PaperPDFCode

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

google-research/semivl officialmentioned in papermentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

DecoderSegmentationSemantic SegmentationSemi-Supervised Semantic Segmentation

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Semi-Supervised Semantic Segmentation ADE20K 1/16 labeled SemiVL Validation mIoU 37.2 #2 of 5 Archive leaderboard report
Semi-Supervised Semantic Segmentation ADE20K 1/32 labeled SemiVL Validation mIoU 35.1 #2 of 5 Archive leaderboard report
Semi-Supervised Semantic Segmentation COCO 1/128 labeled SemiVL Validation mIoU 53.6 #2 of 9 Archive leaderboard report
Semi-Supervised Semantic Segmentation COCO 1/256 labeled SemiVL Validation mIoU 52.8 #2 of 9 Archive leaderboard report
Semi-Supervised Semantic Segmentation COCO 1/32 labeled SemiVL Validation mIoU 56.5 #2 of 7 Archive leaderboard report
Semi-Supervised Semantic Segmentation COCO 1/512 labeled SemiVL Validation mIoU 50.1 #1 of 8 Archive leaderboard report
Semi-Supervised Semantic Segmentation COCO 1/64 labeled SemiVL Validation mIoU 55.4 #2 of 9 Archive leaderboard report
Semi-Supervised Semantic Segmentation Cityscapes 100 samples labeled SemiVL (ViT-B/16) Validation mIoU 76.2 #1 of 13 Archive leaderboard report
Semi-Supervised Semantic Segmentation Cityscapes 12.5% labeled SemiVL (ViT-B/16) Validation mIoU 79.4% #2 of 33 Archive leaderboard report
Semi-Supervised Semantic Segmentation Cityscapes 25% labeled SemiVL (ViT-B/16) Validation mIoU 80.3% #2 of 30 Archive leaderboard report
Semi-Supervised Semantic Segmentation Cityscapes 50% labeled SemiVL (ViT-B/16) Validation mIoU 80.6% #3 of 23 Archive leaderboard report
Semi-Supervised Semantic Segmentation Cityscapes 6.25% labeled SemiVL (ViT-B/16) Validation mIoU 77.9 #2 of 18 Archive leaderboard report
Semi-Supervised Semantic Segmentation PASCAL VOC 2012 1464 labels SemiVL (ViT-B/16 Validation mIoU 87.3 #3 of 17 Archive leaderboard report
Semi-Supervised Semantic Segmentation PASCAL VOC 2012 1464 labels UniMatch (ViT-B/16) Validation mIoU 84.0 #4 of 17 Archive leaderboard report
Semi-Supervised Semantic Segmentation PASCAL VOC 2012 183 labeled SemiVL (ViT-B/16) Validation mIoU 85.6 #3 of 16 Archive leaderboard report
Semi-Supervised Semantic Segmentation PASCAL VOC 2012 183 labeled UniMatch (ViT-B/16) Validation mIoU 80.1 #5 of 16 Archive leaderboard report
Semi-Supervised Semantic Segmentation PASCAL VOC 2012 366 labeled SemiVL (ViT-B/16) Validation mIoU 86.0 #3 of 15 Archive leaderboard report
Semi-Supervised Semantic Segmentation PASCAL VOC 2012 366 labeled UniMatch (ViT-B/16) Validation mIoU 82.0 #4 of 15 Archive leaderboard report
Semi-Supervised Semantic Segmentation PASCAL VOC 2012 732 labeled SemiVL (ViT-B/16) Validation mIoU 86.7 #3 of 16 Archive leaderboard report
Semi-Supervised Semantic Segmentation PASCAL VOC 2012 732 labeled UniMatch (ViT-B/16) Validation mIoU 83.3 #4 of 16 Archive leaderboard report
Semi-Supervised Semantic Segmentation PASCAL VOC 2012 92 labeled SemiVL (ViT-B/16) Validation mIoU 84.0 #3 of 17 Archive leaderboard report
Semi-Supervised Semantic Segmentation PASCAL VOC 2012 92 labeled UniMatch (ViT-B/16) Validation mIoU 77.9 #6 of 17 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections