Papers › Segment Every Reference Object in Spatial and Temporal Spaces

Segment Every Reference Object in Spatial and Temporal Spaces

1 Jan 2023ICCV 2023 1archive 2025-07-28

Jiannan Wu, Yi Jiang, Bin Yan, Huchuan Lu, Zehuan Yuan, Ping Luo

The reference-based object segmentation tasks, namely referring image segmentation (RIS), referring video object segmentation (RVOS), and video object segmentation (VOS), aim to segment a specific object by utilizing either language or annotated masks as references. Despite significant progress in each respective field, current methods are task-specifically designed and developed in different directions, which hinders the activation of multi-task capabilities for these tasks. In this work, we end the current fragmented situation and propose UniRef to unify the three reference-based object segmentation tasks with a single architecture. At the heart of our approach is the multiway-fusion for handling different task with respect to their specified references. And a unified Transformer architecture is then adopted for performing instance-level segmentation. With the unified designs, UniRef can be jointly trained on a broad range of benchmarks and can flexibly perform multiple tasks at runtime by specifying the corresponding references. We evaluate the jointly trained network on various benchmarks. Extensive experimental results indicate that our proposed UniRef achieves state-of-the-art performance on RIS and RVOS, and performs competitively on VOS with a single network.

Paper PDF

Code

No code repository is listed for this paper in the archive or in Syntology's graph.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Image Segmentation Object Referring Expression Segmentation Referring Video Object Segmentation Segmentation Semantic Segmentation Video Object Segmentation Video Semantic Segmentation

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Referring Expression Segmentation	Refer-YouTube-VOS (2021 public validation)	UniRef-L (Swin-L)	F	69.2	#8 of 33	Archive leaderboard	report
Referring Expression Segmentation	Refer-YouTube-VOS (2021 public validation)	UniRef-L (Swin-L)	J	65.5	#8 of 33	Archive leaderboard	report
Referring Expression Segmentation	Refer-YouTube-VOS (2021 public validation)	UniRef-L (Swin-L)	J&F	67.4	#8 of 33	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Absolute Position Encodings Adam Attention BPE Dense Connections Dropout Label Smoothing Layer Normalization Linear Layer Multi-Head Attention Position-Wise Feed-Forward Layer Residual Connection Softmax Transformer VOS

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections