Browse State-of-the-Art › Video Panoptic Segmentation
Video Panoptic Segmentation
21 papers with code · 5 benchmarks · 6 datasets archive 2025-07-28
Video Panoptic Segmentation is a computer vision task that extends panoptic segmentation by incorporating temporal dimension. That is, given a video sequence, the goal is to predict the semantic class of each pixel while consistently tracking object instances. Here, the pixels belonging to the same object instance should be assigned the same instance ID throughout the video sequence.
Description from the archive archive 2025-07-28; Papers-with-Code links inside it are rewritten to this site.
Benchmarks archive 2025-07-28
5 leaderboard tables shown for this task, 5 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| VIPSeg (12 rows) | CAVIS(VIT-L) | Context-Aware Video Instance Segmentation | code | — | Compare |
| Cityscapes-VPS (8 rows) | VIP-Deeplab | ViP-DeepLab: Learning Visual Perception with Depth-aware Video... | code | Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified | Compare |
| KITTI-STEP (6 rows) | Video K-Net (Swin-L) | Video K-Net: A Simple, Strong, and Unified Baseline for Video Segmentation | code | — | Compare |
| 4D-OR (2 rows) | MM-OR-VPQ4 | MM-OR: A Large Multimodal Operating Room Dataset for Semantic... | code | — | Compare |
| MM-OR (2 rows) | MM-OR-VPQ4 | MM-OR: A Large Multimodal Operating Room Dataset for Semantic... | code | — | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
6 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 42 · with a code link: 21 · where Syntology ran a sample: 4 (3 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (4 of 42 tagged: 3 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 42 of 42, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
30 Nov 2023 2 repositories listedIn this work, we present Axial-VS, a general and simple framework that enhances video segmenters by tracking objects along axial trajectories.
-
9 Dec 2020 2 repositories listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 3 pointer-only (licence)We name this joint task as Depth-aware Video Panoptic Segmentation, and propose a new evaluation metric along with two derived datasets for it, which will be made available to the public.
-
4 Mar 2025 1 repository listedOperating rooms (ORs) are complex, high-stakes environments requiring precise understanding of interactions among medical staff, tools, and equipment for enhancing surgical assistance, situational awareness, and patient…
-
3 Jul 2024 1 repository listedIn this paper, we introduce the Context-Aware Video Instance Segmentation (CAVIS), a novel framework designed to enhance instance association by integrating contextual information adjacent to each object.
-
1 Jul 2024 1 repository listedWe present Uni-DVPS, a unified model for Depth-aware Video Panoptic Segmentation (DVPS) that jointly tackles distinct vision tasks, i. e., video panoptic segmentation, monocular depth estimation, and object tracking.
-
16 May 2024 1 repository listedThe findings of the conducted quantitative and qualitative evaluations demonstrate the ability of our framework to spot the most and least influential fragments and visual objects of the video for the summarizer, and to…
-
28 Feb 2024 1 repository listed Syntology official (archive's flag): 12 ran · 12 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 1 violated, 8 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 14 harvested samples) · 14 pointer-only (licence)Despite the recent advances in unified image segmentation (IS), developing a unified video segmentation (VS) model remains a challenge.
-
20 Dec 2023 1 repository listedWe present the Decoupled VIdeo Segmentation (DVIS) framework, a novel approach for the challenging task of universal video segmentation, including video instance segmentation (VIS), video semantic segmentation (VSS)…
-
7 Sep 2023 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 10 harvested samples) · 10 pointer-only (licence)To 'track anything' without training on video data for every individual task, we develop a decoupled video segmentation approach (DEVA), composed of task-specific image-level segmentation and class/task-agnostic…
-
7 Jun 2023 1 repository listedIn this report, we successfully validated the effectiveness of the decoupling strategy in video panoptic segmentation.
-
6 Jun 2023 1 repository listedThe efficacy of the decoupling strategy relies on two crucial elements: 1) attaining precise long-term alignment outcomes via frame-by-frame association during tracking, and 2) the effective utilization of temporal…
-
22 Mar 2023 1 repository listedOur framework is a near-online approach that takes a short subclip as input and outputs the corresponding spatial-temporal tube masks.
-
6 Jan 2023 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 6 harvested samples)A single TarViS model can be trained jointly on a collection of datasets spanning different tasks, and can hot-swap between tasks during inference without any task-specific retraining.
-
1 Jan 2023 1 repository listedWe evaluate the proposed approach across three challenging tasks: video instance segmentation, multi-object tracking and segmentation, and video panoptic segmentation.
-
4 Jul 2022 1 repository listedWe present PVO, a novel panoptic visual odometry framework to achieve more comprehensive modeling of the scene motion, geometry, and panoptic segmentation information.
-
15 Jun 2022 1 repository listedWe therefore present the Waymo Open Dataset: Panoramic Video Panoptic Segmentation Dataset, a large-scale dataset that offers high-quality panoptic segmentation labels for autonomous driving.
-
10 Apr 2022 1 repository listedWe hope this simple, yet effective method can serve as a new, flexible baseline in unified video segmentation design.
-
1 Jan 2022 1 repository listedIn contrast, our large-scale VIdeo Panoptic Segmentation in the Wild (VIPSeg) dataset provides 3, 536 videos and 84, 750 frames with pixel-level panoptic annotations, covering a wide range of real-world scenarios and…
-
5 Dec 2021 1 repository listedThe Depth-aware Video Panoptic Segmentation (DVPS) is a new challenging vision problem that aims to predict panoptic segmentation and depth in a video simultaneously.
-
23 Feb 2021 1 repository listedThe task of assigning semantic classes and track identities to every pixel in a video is called video panoptic segmentation.
-
19 Jun 2020 1 repository listedIn this paper, we propose and explore a new video extension of this task, called video panoptic segmentation.
-
A Comprehensive Survey on Video Scene Parsing:Advances, Challenges, and Prospects16 Jun 2025 0 repositories listedVideo Scene Parsing (VSP) has emerged as a cornerstone in computer vision, facilitating the simultaneous segmentation, recognition, and tracking of diverse visual entities in dynamic scenes.
-
LiDAR-Camera Fusion for Video Panoptic Segmentation without Video Training30 Dec 2024 0 repositories listedPanoptic segmentation, which combines instance and semantic segmentation, has gained a lot of attention in autonomous vehicles, due to its comprehensive representation of the scene.
-
Balancing Shared and Task-Specific Representations: A Hybrid Approach to Depth-Aware Video Panoptic Segmentation10 Dec 2024 0 repositories listedIn this work, we present Multiformer, a novel approach to depth-aware video panoptic segmentation (DVPS) based on the mask transformer paradigm.
-
MGNiceNet: Unified Monocular Geometric Scene Understanding18 Nov 2024 0 repositories listedMonocular geometric scene understanding combines panoptic segmentation and self-supervised depth estimation, focusing on real-time application in autonomous vehicles.
-
Configurable Embodied Data Generation for Class-Agnostic RGB-D Video Segmentation16 Oct 2024 0 repositories listedThis paper presents a method for generating large-scale datasets to improve class-agnostic video segmentation across robots with different form factors.
-
1st Place Winner of the 2024 Pixel-level Video Understanding in the Wild (CVPR'24 PVUW) Challenge in Video Panoptic Segmentation and Best Long Video Consistency of Video Semantic Segmentation8 Jun 2024 0 repositories listedThe third Pixel-level Video Understanding in the Wild (PVUW CVPR 2024) challenge aims to advance the state of art in video understanding through benchmarking Video Panoptic Segmentation (VPS) and Video Semantic…
-
3rd Place Solution for PVUW Challenge 2024: Video Panoptic Segmentation6 Jun 2024 0 repositories listedVideo panoptic segmentation is an advanced task that extends panoptic segmentation by applying its concept to video sequences.
-
2nd Place Solution for PVUW Challenge 2024: Video Panoptic Segmentation1 Jun 2024 0 repositories listedVideo Panoptic Segmentation (VPS) is a challenging task that is extends from image panoptic segmentation.VPS aims to simultaneously classify, track, segment all objects in a video, including both things and stuff.
-
SANPO: A Scene Understanding, Accessibility and Human Navigation Dataset21 Sep 2023 0 repositories listedVision is essential for human navigation.
-
PAg-NeRF: Towards fast and efficient end-to-end panoptic 3D representations for agricultural robotics11 Sep 2023 0 repositories listed
-
3rd Place Solution for PVUW Challenge 2023: Video Panoptic Segmentation11 Jun 2023 0 repositories listed
-
Video-kMaX: A Simple Unified Approach for Online and Near-Online Video Panoptic Segmentation10 Apr 2023 0 repositories listed
-
3 Mar 2023 0 repositories listed
-
MonoDVPS: A Self-Supervised Monocular Depth Estimation Approach to Depth-aware Video Panoptic Segmentation14 Oct 2022 0 repositories listed
-
Time-Space Transformers for Video Panoptic Segmentation7 Oct 2022 0 repositories listed
-
Hybrid Tracker with Pixel and Instance for Video Panoptic Segmentation2 Mar 2022 0 repositories listed
-
Automated processing of X-ray computed tomography images via panoptic segmentation for modeling woven composite textiles2 Feb 2022 0 repositories listed
-
Slot-VPS: Object-centric Representation Learning for Video Panoptic Segmentation16 Dec 2021 0 repositories listed
-
An End-to-End Trainable Video Panoptic Segmentation Method usingTransformers8 Oct 2021 0 repositories listed
-
Merging Tasks for Video Panoptic Segmentation10 Jul 2021 0 repositories listed
-
17 Jun 2021 0 repositories listed
Syntology lines on 4 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections