{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/associating-inter-image-salient-instances-for","title":"Associating Inter-Image Salient Instances for Weakly Supervised Semantic Segmentation","arxiv_id":null,"date":"2018-09-01","proceeding":"ECCV 2018 9","authors":["Ruochen Fan","Qibin Hou","Ming-Ming Cheng","Gang Yu","Ralph R. Martin","Shi-Min Hu"],"abstract":"Effectively bridging between image level keyword annotations and corresponding image pixels is one of the main challenges in weakly supervised semantic segmentation. In this paper, we use an instance-level salient object detector to automatically generate salient instances (candidate objects) for training images. Using similarity features extracted from each salient instance in the whole training set, we build a similarity graph, then use a graph partitioning algorithm to separate it into multiple subgraphs, each of which is associated with a single keyword (tag). Our graph-partitioning-based clustering algorithm allows us to consider the relationships between all salient instances in the training set as well as the information within them. We further show that with the help of attention information, our clustering algorithm is able to correct certain wrong assignments, leading to more accurate results. The proposed framework is general, and any state-of-the-art fully-supervised network structure can be incorporated to learn the segmentation network. When working with DeepLab for semantic segmentation, our method outperforms state-of-the-art weakly supervised alternatives by a large margin, achieving  65.6% mIoU on the PASCAL VOC 2012 dataset. We also combine our method with Mask R-CNN for instance segmentation, and demonstrated for the first time the ability of weakly supervised instance segmentation using only keyword annotations. ","url_abs":"http://openaccess.thecvf.com/content_ECCV_2018/html/Ruochen_Fan_Associating_Inter-Image_Salient_ECCV_2018_paper.html","url_pdf":"http://openaccess.thecvf.com/content_ECCV_2018/papers/Ruochen_Fan_Associating_Inter-Image_Salient_ECCV_2018_paper.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"clustering","task_name":"Clustering"},{"task_slug":"image-level-supervised-instance-segmentation","task_name":"Image-level Supervised Instance Segmentation"},{"task_slug":"instance-segmentation","task_name":"Instance Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"tag","task_name":"TAG"},{"task_slug":"weakly-supervised-semantic-segmentation-1","task_name":"Weakly supervised Semantic Segmentation"},{"task_slug":"weakly-supervised-semantic-segmentation","task_name":"Weakly-Supervised Semantic Segmentation"},{"task_slug":"weakly-supervised-instance-segmentation","task_name":"Weakly-supervised instance segmentation"},{"task_slug":"graph-partitioning","task_name":"graph partitioning"}],"methods":[{"method_slug":"crf","method_name":"CRF"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"deeplab","method_name":"DeepLab"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dilated-convolution","method_name":"Dilated Convolution"},{"method_slug":"feedforward-network","method_name":"Feedforward Network"},{"method_slug":"mask-r-cnn","method_name":"Mask R-CNN"},{"method_slug":"rpn","method_name":"RPN"},{"method_slug":"roi-align","method_name":"RoIAlign"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-level-supervised-instance-segmentation-1","task":"Image-level Supervised Instance Segmentation","dataset":"COCO test-dev","model":"Fan et al.","rank_in_archive_order":6,"of":7,"metrics":{"AP":"13.7","AP@50":"25.5","AP@75":"13.5"},"uses_additional_data":true}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}