Papers › PhraseCut: Language-based Image Segmentation in the Wild

PhraseCut: Language-based Image Segmentation in the Wild

3 Aug 2020CVPR 2020 6arXiv:2008.01187archive 2025-07-28

Chenyun Wu, Zhe Lin, Scott Cohen, Trung Bui, Subhransu Maji

We consider the problem of segmenting image regions given a natural language phrase, and study it on a novel dataset of 77,262 images and 345,486 phrase-region pairs. Our dataset is collected on top of the Visual Genome dataset and uses the existing annotations to generate a challenging set of referring phrases for which the corresponding regions are manually annotated. Phrases in our dataset correspond to multiple regions and describe a large number of object and stuff categories as well as their attributes such as color, shape, parts, and relationships with other entities in the image. Our experiments show that the scale and diversity of concepts in our dataset poses significant challenges to the existing state-of-the-art. We systematically handle the long-tail nature of these concepts and present a modular approach to combine category, attribute, and relationship cues that outperforms existing approaches.

PaperPDFConference PDFCode

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

AttributeDiversityImage SegmentationReferring Expression SegmentationSemantic Segmentation

Datasets

Introduced by this paper, per the archive.

PhraseCut

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Referring Expression Segmentation PhraseCut HULANet Mean IoU 41.3 #4 of 6 Archive leaderboard report
Referring Expression Segmentation PhraseCut HULANet Pr@0.5 42.9 #4 of 6 Archive leaderboard report
Referring Expression Segmentation PhraseCut HULANet Pr@0.7 27.8 #4 of 6 Archive leaderboard report
Referring Expression Segmentation PhraseCut HULANet Pr@0.9 5.9 #4 of 6 Archive leaderboard report
Referring Expression Segmentation PhraseCut RMI Mean IoU 21.1 #5 of 6 Archive leaderboard report
Referring Expression Segmentation PhraseCut RMI Pr@0.5 22 #5 of 6 Archive leaderboard report
Referring Expression Segmentation PhraseCut RMI Pr@0.7 11.6 #5 of 6 Archive leaderboard report
Referring Expression Segmentation PhraseCut RMI Pr@0.9 1.5 #5 of 6 Archive leaderboard report
Referring Expression Segmentation PhraseCut MattNet Mean IoU 20.2 #6 of 6 Archive leaderboard report
Referring Expression Segmentation PhraseCut MattNet Pr@0.5 19.7 #6 of 6 Archive leaderboard report
Referring Expression Segmentation PhraseCut MattNet Pr@0.7 13.5 #6 of 6 Archive leaderboard report
Referring Expression Segmentation PhraseCut MattNet Pr@0.9 3 #6 of 6 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections