{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/self-supervised-learning-of-object-parts-for","title":"Self-Supervised Learning of Object Parts for Semantic Segmentation","arxiv_id":"2204.13101","date":"2022-04-27","proceeding":"CVPR 2022 1","authors":["Adrian Ziegler","Yuki M. Asano"],"abstract":"Progress in self-supervised learning has brought strong general image representation learning methods. Yet so far, it has mostly focused on image-level learning. In turn, tasks such as unsupervised image segmentation have not benefited from this trend as they require spatially-diverse representations. However, learning dense representations is challenging, as in the unsupervised context it is not clear how to guide the model to learn representations that correspond to various potential object categories. In this paper, we argue that self-supervised learning of object parts is a solution to this issue. Object parts are generalizable: they are a priori independent of an object definition, but can be grouped to form objects a posteriori. To this end, we leverage the recently proposed Vision Transformer's capability of attending to objects and combine it with a spatially dense clustering task for fine-tuning the spatial tokens. Our method surpasses the state-of-the-art on three semantic segmentation benchmarks by 17%-3%, showing that our representations are versatile under various object definitions. Finally, we extend this to fully unsupervised segmentation - which refrains completely from using label information even at test-time - and demonstrate that a simple method for automatically merging discovered object parts based on community detection yields substantial gains.","url_abs":"https://arxiv.org/abs/2204.13101v2","url_pdf":"https://arxiv.org/pdf/2204.13101v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"self-supervised-learning-of-object-parts-for","repo_url":"https://github.com/mkuuwaujinga/leopart","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"community-detection","task_name":"Community Detection"},{"task_slug":"image-segmentation","task_name":"Image Segmentation"},{"task_slug":"object","task_name":"Object"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"unsupervised-image-segmentation","task_name":"Unsupervised Image Segmentation"},{"task_slug":"unsupervised-semantic-segmentation","task_name":"Unsupervised Semantic Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/unsupervised-semantic-segmentation-on-pascal-1","task":"Unsupervised Semantic Segmentation","dataset":"PASCAL VOC 2012 val","model":"Leopart (ViT-B/8)","rank_in_archive_order":5,"of":12,"metrics":{"Clustering [mIoU]":"47.2","FCN [mIoU]":"76.3"},"uses_additional_data":true},{"leaderboard":"/sota/unsupervised-semantic-segmentation-on-pascal-1","task":"Unsupervised Semantic Segmentation","dataset":"PASCAL VOC 2012 val","model":"Leopart (ViT-S/16)","rank_in_archive_order":10,"of":12,"metrics":{"Clustering [mIoU]":"41.7","FCN [mIoU]":"71.4","Linear Classifier [mIoU]":"69.3"},"uses_additional_data":true}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2204.13101","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2204.13101"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mkuuwaujinga/leopart","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/MkuuWaUjinga/leopart","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3,"ran_fixture":2,"unverified":4},"by_repo_kind":{"official":{"samples":9,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4a84019b30b6193a","entry":"bfscore","repo":"MkuuWaUjinga/leopart","repo_kind":"official","path":"experiments/bf1_eval/bfscore.py","file_url":"https://github.com/MkuuWaUjinga/leopart/blob/HEAD/experiments/bf1_eval/bfscore.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4a84019b30b6193a"}},{"code_sha256_prefix":"18c24ad4e75b0f50","entry":"bfscore_old","repo":"MkuuWaUjinga/leopart","repo_kind":"official","path":"experiments/bf1_eval/bfscore.py","file_url":"https://github.com/MkuuWaUjinga/leopart/blob/HEAD/experiments/bf1_eval/bfscore.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"18c24ad4e75b0f50"}},{"code_sha256_prefix":"0801a428f3bdc324","entry":"calc_precision_recall","repo":"MkuuWaUjinga/leopart","repo_kind":"official","path":"experiments/bf1_eval/bfscore.py","file_url":"https://github.com/MkuuWaUjinga/leopart/blob/HEAD/experiments/bf1_eval/bfscore.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0801a428f3bdc324"}},{"code_sha256_prefix":"55120f2026b56aa2","entry":"drop_path","repo":"MkuuWaUjinga/leopart","repo_kind":"official","path":"src/vit.py","file_url":"https://github.com/MkuuWaUjinga/leopart/blob/HEAD/src/vit.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"55120f2026b56aa2"}},{"code_sha256_prefix":"056f371660385ea5","entry":"trunc_normal_","repo":"MkuuWaUjinga/leopart","repo_kind":"official","path":"src/vit.py","file_url":"https://github.com/MkuuWaUjinga/leopart/blob/HEAD/src/vit.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"056f371660385ea5"}},{"code_sha256_prefix":"a159b9d724938be2","entry":"compute_features","repo":"MkuuWaUjinga/leopart","repo_kind":"official","path":"experiments/utils.py","file_url":"https://github.com/MkuuWaUjinga/leopart/blob/HEAD/experiments/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a159b9d724938be2"}},{"code_sha256_prefix":"149be6eb5a03c4d3","entry":"eval_jac","repo":"MkuuWaUjinga/leopart","repo_kind":"official","path":"experiments/utils.py","file_url":"https://github.com/MkuuWaUjinga/leopart/blob/HEAD/experiments/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"149be6eb5a03c4d3"}},{"code_sha256_prefix":"1e4abfabdbaf6a81","entry":"get_backbone_weights","repo":"MkuuWaUjinga/leopart","repo_kind":"official","path":"experiments/utils.py","file_url":"https://github.com/MkuuWaUjinga/leopart/blob/HEAD/experiments/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1e4abfabdbaf6a81"}},{"code_sha256_prefix":"c1e452e1b1176484","entry":"vit_tiny","repo":"MkuuWaUjinga/leopart","repo_kind":"official","path":"src/vit.py","file_url":"https://github.com/MkuuWaUjinga/leopart/blob/HEAD/src/vit.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c1e452e1b1176484"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}