Referring Expression Segmentation

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

22 leaderboard tables shown for this task, 22 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted. 10 shown of 22 until expanded.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

11 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

2 subtasks in the archive's task tree.

Parent tasks archive 2025-07-28

Most implemented papers archive 2025-07-28

30 shown of 97 papers with code (145 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Image Segmentation Using Text and Image Prompts
  18 Dec 2021 6 repositories listed Syntology ran 1 of 2 samples · 1 unverified · 2 pointer-only (licence)
  After training on an extended version of the PhraseCut dataset, our system generates a binary segmentation map for an image based on a free-text prompt or on an additional image expressing the query.
• MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding
  26 Apr 2021 5 repositories listed Syntology ran 6 of 11 samples · 5 unverified
  We also investigate the utility of our model as an object detector on a given label set when fine-tuned in a few-shot setting.
• Segmentation from Natural Language Expressions
  20 Mar 2016 4 repositories listed
  To produce pixelwise segmentation for the language expression, we propose an end-to-end trainable recurrent and convolutional network model that jointly learns to process visual and linguistic information.
• VisionReasoner: Unified Visual Perception and Reasoning via Reinforcement Learning
  17 May 2025 3 repositories listed Syntology ran 1 of 15 samples · 14 unverified
  Large vision-language models exhibit inherent capabilities to handle diverse visual perception tasks.
• Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement
  9 Mar 2025 3 repositories listed
  Traditional methods for reasoning segmentation rely on supervised fine-tuning with categorical labels and simple descriptions, limiting its out-of-domain generalization and lacking explicit reasoning processes.
• SeqTR: A Simple yet Universal Network for Visual Grounding
  30 Mar 2022 3 repositories listed Syntology ran 1 of 4 samples · 3 unverified
  In this paper, we propose a simple yet universal network termed SeqTR for visual grounding tasks, e.
• CLEVR-Ref+: Diagnosing Visual Reasoning with Referring Expressions
  3 Jan 2019 3 repositories listed Syntology ran 4 of 5 samples · 1 unverified · 5 pointer-only (licence)
  Yet there has been evidence that current benchmark datasets suffer from bias, and current state-of-the-art models cannot be easily evaluated on their intermediate reasoning process.
• 3D-GRES: Generalized 3D Referring Expression Segmentation
  30 Jul 2024 2 repositories listed Syntology ran 4 of 9 samples · 5 unverified · 9 pointer-only (licence)
  3D Referring Expression Segmentation (3D-RES) is dedicated to segmenting a specific instance within a 3D space based on a natural language description.
• F-LMM: Grounding Frozen Large Multimodal Models
  9 Jun 2024 2 repositories listed
  To address this issue, we present F-LMM -- grounding frozen off-the-shelf LMMs in human-AI conversations -- a straightforward yet effective design based on the fact that word-pixel correspondences conducive to visual…
• CoHD: A Counting-Aware Hierarchical Decoding Framework for Generalized Referring Expression Segmentation
  24 May 2024 2 repositories listed
  By decoupling the intricate referring semantics into different granularity with a visual-linguistic hierarchy, and dynamic aggregating it with intra- and inter-selection, CoHD boosts multi-granularity comprehension with…
• UniRef++: Segment Every Reference Object in Spatial and Temporal Spaces
  25 Dec 2023 2 repositories listed Syntology ran 9 of 9 samples · 0 unverified
  We evaluate our unified models on various benchmarks.
• Universal Segmentation at Arbitrary Granularity with Language Instruction
  4 Dec 2023 2 repositories listed Syntology ran 13 of 16 samples · 3 unverified
  This paper aims to achieve universal segmentation of arbitrary semantic level.
• Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
  24 Aug 2023 2 repositories listed Syntology ran 0 of 2 samples · 2 unverified · 2 pointer-only (licence)
  In this work, we introduce the Qwen-VL series, a set of large-scale vision-language models (LVLMs) designed to perceive and understand both texts and images.
• GRES: Generalized Referring Expression Segmentation
  1 Jun 2023 2 repositories listed Syntology ran 2 of 5 samples · 3 unverified
  Existing classic RES datasets and methods commonly support single-target expressions only, i.
• Unleashing Text-to-Image Diffusion Models for Visual Perception
  3 Mar 2023 2 repositories listed Syntology ran 4 of 5 samples · 1 unverified
  In this paper, we propose VPD (Visual Perception with a pre-trained Diffusion model), a new framework that exploits the semantic information of a pre-trained text-to-image diffusion model in visual perception tasks.
• End-to-End Referring Video Object Segmentation with Multimodal Transformers
  29 Nov 2021 2 repositories listed Syntology ran 6 of 11 samples · 5 unverified
  Due to the complex nature of this multimodal task, which combines text reasoning, video understanding, instance segmentation and tracking, existing approaches typically rely on sophisticated pipelines in order to tackle…
• SynthRef: Generation of Synthetic Referring Expressions for Object Segmentation
  8 Jun 2021 2 repositories listed
  Recent advances in deep learning have brought significant progress in visual grounding tasks such as language-guided video object segmentation.
• RefVOS: A Closer Look at Referring Expressions for Video Object Segmentation
  1 Oct 2020 2 repositories listed
  The task of video object segmentation with referring expressions (language-guided VOS) is to, given a linguistic phrase and a video, generate binary masks for the object to which the phrase refers.
• Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentation
  19 Mar 2020 2 repositories listed Syntology ran 0 of 13 samples · 13 unverified
  In addition, we address a key challenge in this multi-task setup, i.
• DeRIS: Decoupling Perception and Cognition for Enhanced Referring Image Segmentation through Loopback Synergy
  2 Jul 2025 1 repository listed
  Referring Image Segmentation (RIS) is a challenging task that aims to segment objects in an image based on natural language expressions.
• RemoteSAM: Towards Segment Anything for Earth Observation
  23 May 2025 1 repository listed
  Combining these innovations on data and modeling, we present RemoteSAM, a foundation model that establishes new SoTA on several earth observation perception benchmarks, outperforming other foundation models such as…
• GroundingSuite: Measuring Complex Multi-Granular Pixel Grounding
  13 Mar 2025 1 repository listed
  Pixel grounding, encompassing tasks such as Referring Expression Segmentation (RES), has garnered considerable attention due to its immense potential for bridging the gap between vision and language modalities.
• SegAgent: Exploring Pixel Understanding Capabilities in MLLMs by Imitating Human Annotator Trajectories
  11 Mar 2025 1 repository listed
  While MLLMs have demonstrated adequate image understanding capabilities, they still struggle with pixel-level comprehension, limiting their practical applications.
• PixFoundation: Are We Heading in the Right Direction with Pixel-level Vision Foundation Models?
  6 Feb 2025 1 repository listed
  In this work, we propose two novel challenging benchmarks and show that MLLMs without pixel-level grounding supervision can outperform the state of the art in such tasks when evaluating both the pixel-level grounding…
• MPG-SAM 2: Adapting SAM 2 with Mask Priors and Global Context for Referring Video Object Segmentation
  23 Jan 2025 1 repository listed Syntology ran 5 of 16 samples · 11 unverified
  Referring video object segmentation (RVOS) aims to segment objects in a video according to textual descriptions, which requires the integration of multimodal information and temporal dynamics perception.
• InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling
  21 Jan 2025 1 repository listed
  This paper aims to improve the performance of video multimodal large language models (MLLM) via long and rich context (LRC) modeling.
• The Devil is in Temporal Token: High Quality Video Reasoning Segmentation
  15 Jan 2025 1 repository listed
  Existing methods for Video Reasoning Segmentation rely heavily on a single special token to represent the object in the keyframe or the entire video, inadequately capturing spatial complexity and inter-frame motion.
• Densely Connected Parameter-Efficient Tuning for Referring Image Segmentation
  15 Jan 2025 1 repository listed Syntology ran 7 of 17 samples · 10 unverified
  While some pioneering studies have undertaken preliminary explorations, they still remain at the level of aligned encoders (e.
• Multi-task Visual Grounding with Coarse-to-Fine Consistency Constraints
  12 Jan 2025 1 repository listed Syntology ran 5 of 13 samples · 8 unverified
  Furthermore, to address the challenge of insufficient multimodal understanding, we leverage pre-trained models based on visual-linguistic fusion representations.
• IPDN: Image-enhanced Prompt Decoding Network for 3D Referring Expression Segmentation
  9 Jan 2025 1 repository listed
  To tackle intent ambiguity, we designed a Prompt-Aware Decoder (PAD) that guides the decoding process by deriving task-driven signals from the interaction between the expression and visual features.

Syntology lines on 16 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.