{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/real-time-human-centric-segmentation-for","title":"Real-time Human-Centric Segmentation for Complex Video Scenes","arxiv_id":"2108.07199","date":"2021-08-16","proceeding":null,"authors":["Ran Yu","Chenyu Tian","Weihao Xia","Xinyuan Zhao","Haoqian Wang","Yujiu Yang"],"abstract":"Most existing video tasks related to \"human\" focus on the segmentation of salient humans, ignoring the unspecified others in the video. Few studies have focused on segmenting and tracking all humans in a complex video, including pedestrians and humans of other states (e.g., seated, riding, or occluded). In this paper, we propose a novel framework, abbreviated as HVISNet, that segments and tracks all presented people in given videos based on a one-stage detector. To better evaluate complex scenes, we offer a new benchmark called HVIS (Human Video Instance Segmentation), which comprises 1447 human instance masks in 805 high-resolution videos in diverse scenes. Extensive experiments show that our proposed HVISNet outperforms the state-of-the-art methods in terms of accuracy at a real-time inference speed (30 FPS), especially on complex video scenes. We also notice that using the center of the bounding box to distinguish different individuals severely deteriorates the segmentation accuracy, especially in heavily occluded conditions. This common phenomenon is referred to as the ambiguous positive samples problem. To alleviate this problem, we propose a mechanism named Inner Center Sampling to improve the accuracy of instance segmentation. Such a plug-and-play inner center sampling mechanism can be incorporated in any instance segmentation models based on a one-stage detector to improve the performance. In particular, it gains 4.1 mAP improvement on the state-of-the-art method in the case of occluded humans. Code and data are available at https://github.com/IIGROUP/HVISNet.","url_abs":"https://arxiv.org/abs/2108.07199v1","url_pdf":"https://arxiv.org/pdf/2108.07199v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"real-time-human-centric-segmentation-for","repo_url":"https://github.com/iigroup/hvisnet","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"human-instance-segmentation","task_name":"Human Instance Segmentation"},{"task_slug":"instance-segmentation","task_name":"Instance Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"video-instance-segmentation","task_name":"Video Instance Segmentation"}],"methods":[],"datasets_introduced":[{"slug":"hvis","name":"HVIS Dataset","full_name":"Human Video Instance Segmentation Dataset"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/human-instance-segmentation-on-ochuman","task":"Human Instance Segmentation","dataset":"OCHuman","model":"BlendMask + CIS","rank_in_archive_order":4,"of":18,"metrics":{"AP":"29.8"},"uses_additional_data":false},{"leaderboard":"/sota/human-instance-segmentation-on-ochuman","task":"Human Instance Segmentation","dataset":"OCHuman","model":"CondInst + CIS","rank_in_archive_order":6,"of":18,"metrics":{"AP":"28.1"},"uses_additional_data":false},{"leaderboard":"/sota/human-instance-segmentation-on-ochuman","task":"Human Instance Segmentation","dataset":"OCHuman","model":"PolarMask + CIS","rank_in_archive_order":14,"of":18,"metrics":{"AP":"23.4"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}