Papers › Decoupling Zero-Shot Semantic Segmentation

Decoupling Zero-Shot Semantic Segmentation

15 Dec 2021CVPR 2022 1arXiv:2112.07910archive 2025-07-28

Jian Ding, Nan Xue, Gui-Song Xia, Dengxin Dai

Zero-shot semantic segmentation (ZS3) aims to segment the novel categories that have not been seen in the training. Existing works formulate ZS3 as a pixel-level zeroshot classification problem, and transfer semantic knowledge from seen classes to unseen ones with the help of language models pre-trained only with texts. While simple, the pixel-level ZS3 formulation shows the limited capability to integrate vision-language models that are often pre-trained with image-text pairs and currently demonstrate great potential for vision tasks. Inspired by the observation that humans often perform segment-level semantic labeling, we propose to decouple the ZS3 into two sub-tasks: 1) a classagnostic grouping task to group the pixels into segments. 2) a zero-shot classification task on segments. The former task does not involve category information and can be directly transferred to group pixels for unseen classes. The latter task performs at segment-level and provides a natural way to leverage large-scale vision-language models pre-trained with image-text pairs (e.g. CLIP) for ZS3. Based on the decoupling formulation, we propose a simple and effective zero-shot semantic segmentation model, called ZegFormer, which outperforms the previous methods on ZS3 standard benchmarks by large margins, e.g., 22 points on the PASCAL VOC and 3 points on the COCO-Stuff in terms of mIoU for unseen classes. Code will be released at https://github.com/dingjiansw101/ZegFormer.

Paper PDF Conference PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

dingjiansw101/zegformer officialmentioned in papermentioned on GitHubpytorchNOASSERTION report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Open Vocabulary Semantic Segmentation Segmentation Semantic Segmentation Zero-Shot Learning Zero-Shot Semantic Segmentation

1 archive task tag without a task page not shown.

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Open Vocabulary Semantic Segmentation	COCO-Stuff-171	ZegFormer	HIoU	34.8	#3 of 7	Archive leaderboard	report
Open Vocabulary Semantic Segmentation	PascalVOC-20	ZegFormer	hIoU	73.3	#20 of 20	Archive leaderboard	report
Zero-Shot Semantic Segmentation	COCO-Stuff	ZegFormer	Inductive Setting hIoU	33.2	#14 of 15	Archive leaderboard	report
Zero-Shot Semantic Segmentation	COCO-Stuff	ZegFormer	Transductive Setting hIoU	-	#14 of 15	Archive leaderboard	report
Zero-Shot Semantic Segmentation	PASCAL VOC	ZegFormer	Inductive Setting hIoU	73.3	#13 of 13	Archive leaderboard	report
Zero-Shot Semantic Segmentation	PASCAL VOC	ZegFormer	Transductive Setting hIoU	-	#13 of 13	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections