{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/leveraging-instance-image-and-dataset-level","title":"Leveraging Instance-, Image- and Dataset-Level Information for Weakly Supervised Instance Segmentation","arxiv_id":null,"date":"2020-09-10","proceeding":null,"authors":["Yun Liu","Yu-Huan Wu","Pei-Song Wen","Yu-Jun Shi","Yu Qiu","Ming-Ming Cheng"],"abstract":"Weakly supervised semantic instance segmentation with only image-level supervision, instead of relying on expensive pixel wise masks or bounding box annotations, is an important problem to alleviate the data-hungry nature of deep learning. In this paper, we tackle this challenging problem by aggregating the image-level information of all training images into a large knowledge graph and exploiting semantic relationships from this graph. Specifically, our effort starts with some generic segment-based object proposals (SOP) without category priors. We propose a multiple instance learning (MIL) framework, which can be trained in an end-to-end manner using training images with image-level labels. For each proposal, this MIL framework can simultaneously compute probability distributions and category-aware semantic features, with which we can formulate a large undirected graph. The category of background is also included in this graph to remove the massive noisy object proposals. An optimal multi-way cut of this graph can thus assign a reliable category label to each proposal. The denoised SOP with assigned category labels can be viewed as pseudo instance segmentation of training images, which are used to train fully supervised models. The proposed approach achieves state-of-the-art performance for both weakly supervised instance segmentation and semantic segmentation.","url_abs":"https://ieeexplore.ieee.org/abstract/document/9193980","url_pdf":"http://mftp.mmcheng.net/Papers/21PAMI_InsImgDatasetWSIS.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"leveraging-instance-image-and-dataset-level","repo_url":"https://github.com/yun-liu/LIID","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-level-supervised-instance-segmentation","task_name":"Image-level Supervised Instance Segmentation"},{"task_slug":"instance-segmentation","task_name":"Instance Segmentation"},{"task_slug":"multiple-instance-learning","task_name":"Multiple Instance Learning"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"weakly-supervised-semantic-segmentation","task_name":"Weakly-Supervised Semantic Segmentation"},{"task_slug":"weakly-supervised-instance-segmentation","task_name":"Weakly-supervised instance segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-level-supervised-instance-segmentation-1","task":"Image-level Supervised Instance Segmentation","dataset":"COCO test-dev","model":"LIID","rank_in_archive_order":4,"of":7,"metrics":{"AP":"16.0","AP@50":"27.1","AP@75":"16.5"},"uses_additional_data":true},{"leaderboard":"/sota/image-level-supervised-instance-segmentation","task":"Image-level Supervised Instance Segmentation","dataset":"PASCAL VOC 2012 val","model":"LIID","rank_in_archive_order":7,"of":13,"metrics":{"mAP@0.5":"48.4","mAP@0.75":"24.9"},"uses_additional_data":true},{"leaderboard":"/sota/weakly-supervised-semantic-segmentation-on-1","task":"Weakly-Supervised Semantic Segmentation","dataset":"PASCAL VOC 2012 test","model":"LIID","rank_in_archive_order":57,"of":60,"metrics":{"Mean IoU":"67.5"},"uses_additional_data":false},{"leaderboard":"/sota/weakly-supervised-semantic-segmentation-on","task":"Weakly-Supervised Semantic Segmentation","dataset":"PASCAL VOC 2012 val","model":"LIID (Res2Net-101)","rank_in_archive_order":49,"of":73,"metrics":{"Mean IoU":"69.4"},"uses_additional_data":false},{"leaderboard":"/sota/weakly-supervised-semantic-segmentation-on","task":"Weakly-Supervised Semantic Segmentation","dataset":"PASCAL VOC 2012 val","model":"LIID (ResNet-101, +24K SI)","rank_in_archive_order":60,"of":73,"metrics":{"Mean IoU":"67.8"},"uses_additional_data":true},{"leaderboard":"/sota/weakly-supervised-semantic-segmentation-on","task":"Weakly-Supervised Semantic Segmentation","dataset":"PASCAL VOC 2012 val","model":"LIID (ResNet-101)","rank_in_archive_order":66,"of":73,"metrics":{"Mean IoU":"66.5"},"uses_additional_data":false},{"leaderboard":"/sota/weakly-supervised-instance-segmentation-on","task":"Weakly-supervised instance segmentation","dataset":"PASCAL VOC 2012 val","model":"LIID","rank_in_archive_order":2,"of":6,"metrics":{"Average Best Overlap":"50.8","mAP@0.25":"-","mAP@0.5":"48.4","mAP@0.75":"24.9"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}