{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/video-panoptic-segmentation-1","title":"Video Panoptic Segmentation","arxiv_id":"2006.11339","date":"2020-06-19","proceeding":"CVPR 2020 6","authors":["Dahun Kim","Sanghyun Woo","Joon-Young Lee","In So Kweon"],"abstract":"Panoptic segmentation has become a new standard of visual recognition task by unifying previous semantic segmentation and instance segmentation tasks in concert. In this paper, we propose and explore a new video extension of this task, called video panoptic segmentation. The task requires generating consistent panoptic segmentation as well as an association of instance ids across video frames. To invigorate research on this new task, we present two types of video panoptic datasets. The first is a re-organization of the synthetic VIPER dataset into the video panoptic format to exploit its large-scale pixel annotations. The second is a temporal extension on the Cityscapes val. set, by providing new video panoptic annotations (Cityscapes-VPS). Moreover, we propose a novel video panoptic segmentation network (VPSNet) which jointly predicts object classes, bounding boxes, masks, instance id tracking, and semantic segmentation in video frames. To provide appropriate metrics for this task, we propose a video panoptic quality (VPQ) metric and evaluate our method and several other baselines. Experimental results demonstrate the effectiveness of the presented two datasets. We achieve state-of-the-art results in image PQ on Cityscapes and also in VPQ on Cityscapes-VPS and VIPER datasets. The datasets and code are made publicly available.","url_abs":"https://arxiv.org/abs/2006.11339v1","url_pdf":"https://arxiv.org/pdf/2006.11339v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"video-panoptic-segmentation-1","repo_url":"https://github.com/mcahny/vps","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"instance-segmentation","task_name":"Instance Segmentation"},{"task_slug":"panoptic-segmentation","task_name":"Panoptic Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"video-instance-segmentation","task_name":"Video Instance Segmentation"},{"task_slug":"video-panoptic-segmentation","task_name":"Video Panoptic Segmentation"},{"task_slug":"video-recognition","task_name":"Video Recognition"},{"task_slug":"video-segmentation","task_name":"Video Segmentation"},{"task_slug":"video-semantic-segmentation","task_name":"Video Semantic Segmentation"}],"methods":[{"method_slug":"vpsnet","method_name":"VPSNet"}],"datasets_introduced":[{"slug":"cityscapes-vps","name":"Cityscapes-VPS","full_name":""}],"methods_introduced":[{"slug":"vpsnet","name":"VPSNet","full_name":"Video Panoptic Segmentation Network"}],"results":[{"leaderboard":"/sota/video-panoptic-segmentation-on-cityscapes-vps","task":"Video Panoptic Segmentation","dataset":"Cityscapes-VPS","model":"VPSNet","rank_in_archive_order":7,"of":8,"metrics":{"VPQ":"57.0","VPQ (stuff)":"66.0","VPQ (thing)":"44.7"},"uses_additional_data":true}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2006.11339","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}