{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/hierarchical-context-learning-of-object","title":"Hierarchical Context Learning of object components for unsupervised semantic segmentation","arxiv_id":null,"date":"2025-04-29","proceeding":"Pattern Recognition 2025 4","authors":["Dong Bao","Jun Zhou","Gervase Tuxworth","Jue Zhang","Yongsheng Gao"],"abstract":"Unsupervised Semantic Segmentation (USS) aims to learn semantically rich and dense representations without\r\nrelying on labels. Recent advances in self-supervised learning have demonstrated the potential of pretrained\r\nvision transformers to capture patch-level semantic information, offering a promising direction to USS.\r\nHowever, existing methods face challenges in constructing a discriminative spatial token embedding space that\r\nconsistently and effectively represents the well-structured semantic relationships among object components.\r\nInspired by Edwin Hancock’s pioneer work on hierarchical pattern analysis, we highlight the critical role\r\nof hierarchical context to overcome this limitation. By modeling spatial relationships at multiple levels of\r\ngranularity, hierarchical context helps align related object parts while distinguishing them across semantic\r\ngroups. Based on this insight, we introduce Hierarchical Context Learning (HCL), a novel approach for USS that\r\nenhances semantic consistency by integrating hierarchical context. HCL incorporates a novel parallel multi-level\r\nvision transformer backbone to aggregate multi-level contextual information into object component tokens.\r\nTo uncover the semantic structure of objects, we propose Momentum-based Global Foreground–Background\r\nClustering (MoGoClustering) to cluster object components into coherent semantic groups and then calculate\r\ntheir semantic centroids. To enforce intra-group semantic consistency and maximize inter-group separation\r\nacross spatial scales, we design a foreground–background-aware contrastive loss based on MoGoClustering.\r\nOur method achieves state-of-the-art performance on the COCO-Stuff and Pascal VOC datasets, demonstrating\r\nits ability to learn robust, context-aware, and discriminative object component semantics for USS. The code is\r\navailable at: https://github.com/dbaofd/HCL.","url_abs":"https://www.sciencedirect.com/science/article/pii/S0031320325003735","url_pdf":"https://www.sciencedirect.com/science/article/pii/S0031320325003735/pdfft?md5=e66df299a16c22511c4226eb0f984d76&pid=1-s2.0-S0031320325003735-main.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"hierarchical-context-learning-of-object","repo_url":"https://github.com/dbaofd/HCL","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"object","task_name":"Object"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"unsupervised-semantic-segmentation","task_name":"Unsupervised Semantic Segmentation"}],"methods":[{"method_slug":"align","method_name":"ALIGN"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/unsupervised-semantic-segmentation-on-coco-7","task":"Unsupervised Semantic Segmentation","dataset":"COCO-Stuff-27","model":"HCL (ViT-S/8)","rank_in_archive_order":28,"of":29,"metrics":{"Linear Classifier [Accuracy]":"77.3","Linear Classifier [mIoU]":"47.4"},"uses_additional_data":false},{"leaderboard":"/sota/unsupervised-semantic-segmentation-on-coco-7","task":"Unsupervised Semantic Segmentation","dataset":"COCO-Stuff-27","model":"HCL (ViT-S/16)","rank_in_archive_order":29,"of":29,"metrics":{"Linear Classifier [Accuracy]":"76.7","Linear Classifier [mIoU]":"45.8"},"uses_additional_data":false},{"leaderboard":"/sota/unsupervised-semantic-segmentation-on-pascal-1","task":"Unsupervised Semantic Segmentation","dataset":"PASCAL VOC 2012 val","model":"HCL (ViT-S/8)","rank_in_archive_order":6,"of":12,"metrics":{"Clustering [mIoU]":"46.3","Linear Classifier [mIoU]":"75.7"},"uses_additional_data":true},{"leaderboard":"/sota/unsupervised-semantic-segmentation-on-pascal-1","task":"Unsupervised Semantic Segmentation","dataset":"PASCAL VOC 2012 val","model":"HCL (ViT-S/16)","rank_in_archive_order":9,"of":12,"metrics":{"Clustering [mIoU]":"43.2","Linear Classifier [mIoU]":"72.6"},"uses_additional_data":true}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}