{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unsupervised-semantic-segmentation-by","title":"Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals","arxiv_id":"2102.06191","date":"2021-02-11","proceeding":"ICCV 2021 10","authors":["Wouter Van Gansbeke","Simon Vandenhende","Stamatios Georgoulis","Luc van Gool"],"abstract":"Being able to learn dense semantic representations of images without supervision is an important problem in computer vision. However, despite its significance, this problem remains rather unexplored, with a few exceptions that considered unsupervised semantic segmentation on small-scale datasets with a narrow visual domain. In this paper, we make a first attempt to tackle the problem on datasets that have been traditionally utilized for the supervised case. To achieve this, we introduce a two-step framework that adopts a predetermined mid-level prior in a contrastive optimization objective to learn pixel embeddings. This marks a large deviation from existing works that relied on proxy tasks or end-to-end clustering. Additionally, we argue about the importance of having a prior that contains information about objects, or their parts, and discuss several possibilities to obtain such a prior in an unsupervised manner. Experimental evaluation shows that our method comes with key advantages over existing works. First, the learned pixel embeddings can be directly clustered in semantic groups using K-Means on PASCAL. Under the fully unsupervised setting, there is no precedent in solving the semantic segmentation task on such a challenging benchmark. Second, our representations can improve over strong baselines when transferred to new datasets, e.g. COCO and DAVIS. The code is available.","url_abs":"https://arxiv.org/abs/2102.06191v3","url_pdf":"https://arxiv.org/pdf/2102.06191v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unsupervised-semantic-segmentation-by","repo_url":"https://github.com/wvangansbeke/Unsupervised-Semantic-Segmentation","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"unsupervised-semantic-segmentation-by","repo_url":"https://github.com/khangt1k25/Contrastive-Segmentation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"clustering","task_name":"Clustering"},{"task_slug":"object","task_name":"Object"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"unsupervised-pre-training","task_name":"Unsupervised Pre-training"},{"task_slug":"unsupervised-semantic-segmentation","task_name":"Unsupervised Semantic Segmentation"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"aspp","method_name":"ASPP"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"deeplabv3","method_name":"DeepLabv3"},{"method_slug":"dilated-convolution","method_name":"Dilated Convolution"},{"method_slug":"spatial-pyramid-pooling","method_name":"Spatial Pyramid Pooling"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/unsupervised-semantic-segmentation-on-coco-8","task":"Unsupervised Semantic Segmentation","dataset":"COCO-Stuff-81","model":"MaskContrast (ResNet-50)","rank_in_archive_order":4,"of":4,"metrics":{"Pixel Accuracy":"8.8","mIoU":"3.7"},"uses_additional_data":false},{"leaderboard":"/sota/unsupervised-semantic-segmentation-on-6","task":"Unsupervised Semantic Segmentation","dataset":"ImageNet-S-50","model":"MaskContrast (+Saliency map)","rank_in_archive_order":3,"of":5,"metrics":{"mIoU (test)":"24.2","mIoU (val)":"24.6"},"uses_additional_data":false},{"leaderboard":"/sota/unsupervised-semantic-segmentation-on-pascal-1","task":"Unsupervised Semantic Segmentation","dataset":"PASCAL VOC 2012 val","model":"MaskContrast (Saliency)","rank_in_archive_order":8,"of":12,"metrics":{"Clustering [mIoU]":"44.2","Linear Classifier [mIoU]":"63.9"},"uses_additional_data":true},{"leaderboard":"/sota/unsupervised-semantic-segmentation-on-pascal-1","task":"Unsupervised Semantic Segmentation","dataset":"PASCAL VOC 2012 val","model":"MaskContrast","rank_in_archive_order":11,"of":12,"metrics":{"Clustering [mIoU]":"35.0","Linear Classifier [mIoU]":"58.4"},"uses_additional_data":true}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2102.06191","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2102.06191"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wvangansbeke/Unsupervised-Semantic-Segmentation","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/khangt1k25/Contrastive-Segmentation","reach":null}],"summary":{"ran":2},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1},"listed":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"6784f6afe0c60b06","entry":"ContrastiveSegmentationModel","repo":"wvangansbeke/Unsupervised-Semantic-Segmentation","repo_kind":"official","path":"segmentation/models/models.py","file_url":"https://github.com/wvangansbeke/Unsupervised-Semantic-Segmentation/blob/HEAD/segmentation/models/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"6784f6afe0c60b06"}},{"code_sha256_prefix":"4a6dcfbed0be0fb1","entry":"ContrastiveSegmentationModel","repo":"khangt1k25/Contrastive-Segmentation","repo_kind":"listed","path":"segmentation/models/models.py","file_url":"https://github.com/khangt1k25/Contrastive-Segmentation/blob/HEAD/segmentation/models/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"4a6dcfbed0be0fb1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}