Referring Expression

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

3 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 166 papers with code (364 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection
  9 Mar 2023 10 repositories listed Syntology ran 2 of 5 samples · 3 unverified
  To effectively fuse language and vision modalities, we conceptually divide a closed-set detector into three phases and propose a tight fusion solution, which includes a feature enhancer, a language-guided query…
• UNITER: UNiversal Image-TExt Representation Learning
  25 Sep 2019 7 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 2 pointer-only (licence)
  Different from previous work that applies joint random masking to both modalities, we use conditional masking on pre-training tasks (i.
• Image Segmentation Using Text and Image Prompts
  18 Dec 2021 6 repositories listed Syntology ran 1 of 2 samples · 1 unverified · 2 pointer-only (licence)
  After training on an extended version of the PhraseCut dataset, our system generates a binary segmentation map for an image based on a free-text prompt or on an additional image expressing the query.
• MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding
  26 Apr 2021 5 repositories listed Syntology ran 6 of 11 samples · 5 unverified
  We also investigate the utility of our model as an object detector on a given label set when fine-tuned in a few-shot setting.
• Towards Visual Grounding: A Survey
  28 Dec 2024 4 repositories listed Syntology ran 9 of 15 samples · 6 unverified · 3 pointer-only (licence)
  Finally, we outline the challenges confronting visual grounding and propose valuable directions for future research, which may serve as inspiration for subsequent researchers.
• Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V
  17 Oct 2023 4 repositories listed Syntology ran 0 of 1 samples · 1 unverified
  We present Set-of-Mark (SoM), a new visual prompting method, to unleash the visual grounding abilities of large multimodal models (LMMs), such as GPT-4V.
• OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework
  7 Feb 2022 4 repositories listed Syntology ran 1 of 1 samples · 0 unverified
  In this work, we pursue a unified paradigm for multimodal pretraining to break the scaffolds of complex task/modality-specific customization.
• Modeling Context in Referring Expressions
  31 Jul 2016 4 repositories listed Syntology ran 1 of 4 samples · 3 unverified
  Humans refer to objects in their environments all the time, especially in dialogue with other people.
• SeqTR: A Simple yet Universal Network for Visual Grounding
  30 Mar 2022 3 repositories listed Syntology ran 1 of 4 samples · 3 unverified
  In this paper, we propose a simple yet universal network termed SeqTR for visual grounding tasks, e.
• VL-BERT: Pre-training of Generic Visual-Linguistic Representations
  22 Aug 2019 3 repositories listed Syntology ran 0 of 1 samples · 1 unverified
  We introduce a new pre-trainable generic representation for visual-linguistic tasks, called Visual-Linguistic BERT (VL-BERT for short).
• CLEVR-Ref+: Diagnosing Visual Reasoning with Referring Expressions
  3 Jan 2019 3 repositories listed Syntology ran 4 of 5 samples · 1 unverified · 5 pointer-only (licence)
  Yet there has been evidence that current benchmark datasets suffer from bias, and current state-of-the-art models cannot be easily evaluated on their intermediate reasoning process.
• 3D-GRES: Generalized 3D Referring Expression Segmentation
  30 Jul 2024 2 repositories listed Syntology ran 4 of 9 samples · 5 unverified · 9 pointer-only (licence)
  3D Referring Expression Segmentation (3D-RES) is dedicated to segmenting a specific instance within a 3D space based on a natural language description.
• F-LMM: Grounding Frozen Large Multimodal Models
  9 Jun 2024 2 repositories listed
  To address this issue, we present F-LMM -- grounding frozen off-the-shelf LMMs in human-AI conversations -- a straightforward yet effective design based on the fact that word-pixel correspondences conducive to visual…
• CoHD: A Counting-Aware Hierarchical Decoding Framework for Generalized Referring Expression Segmentation
  24 May 2024 2 repositories listed
  By decoupling the intricate referring semantics into different granularity with a visual-linguistic hierarchy, and dynamic aggregating it with intra- and inter-selection, CoHD boosts multi-granularity comprehension with…
• LLMs as Bridges: Reformulating Grounded Multimodal Named Entity Recognition
  15 Feb 2024 2 repositories listed
  Grounded Multimodal Named Entity Recognition (GMNER) is a nascent multimodal task that aims to identify named entities, entity types and their corresponding visual regions.
• An Open and Comprehensive Pipeline for Unified Object Grounding and Detection
  4 Jan 2024 2 repositories listed Syntology ran 4 of 6 samples · 2 unverified
  Grounding-DINO is a state-of-the-art open-set detection model that tackles multiple vision tasks including Open-Vocabulary Detection (OVD), Phrase Grounding (PG), and Referring Expression Comprehension (REC).
• Localized Symbolic Knowledge Distillation for Visual Commonsense Models
  8 Dec 2023 2 repositories listed Syntology ran 8 of 10 samples · 2 unverified
  Empirical results and human evaluations in a zero-shot setup demonstrate that our distillation method results in more precise VL models of reasoning compared to a baseline of passing a generated referring expression to…
• Kosmos-2: Grounding Multimodal Large Language Models to the World
  26 Jun 2023 2 repositories listed
  We introduce Kosmos-2, a Multimodal Large Language Model (MLLM), enabling new capabilities of perceiving object descriptions (e.
• GRES: Generalized Referring Expression Segmentation
  1 Jun 2023 2 repositories listed Syntology ran 2 of 5 samples · 3 unverified
  Existing classic RES datasets and methods commonly support single-target expressions only, i.
• ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehension
  12 Apr 2022 2 repositories listed Syntology ran 0 of 5 samples · 5 unverified
  Training a referring expression comprehension (ReC) model for a new visual domain requires collecting referring expressions, and potentially corresponding bounding boxes, for images in the domain.
• The Project Dialogism Novel Corpus: A Dataset for Quotation Attribution in Literary Texts
  12 Apr 2022 2 repositories listed
  We present the Project Dialogism Novel Corpus, or PDNC, an annotated dataset of quotations for English literary texts.
• Airbert: In-domain Pretraining for Vision-and-Language Navigation
  20 Aug 2021 2 repositories listed Syntology ran 7 of 14 samples · 7 unverified
  Given the scarcity of domain-specific training data and the high diversity of image and language inputs, the generalization of VLN agents to unseen environments remains challenging.
• Unifying Vision-and-Language Tasks via Text Generation
  4 Feb 2021 2 repositories listed Syntology ran 2 of 12 samples · 10 unverified
  On 7 popular vision-and-language benchmarks, including visual question answering, referring expression comprehension, visual commonsense reasoning, most of which have been previously modeled as discriminative tasks, our…
• Large-Scale Adversarial Training for Vision-and-Language Representation Learning
  11 Jun 2020 2 repositories listed Syntology ran 10 of 20 samples · 10 unverified · 6 pointer-only (licence)
  We present VILLA, the first known effort on large-scale adversarial training for vision-and-language (V+L) representation learning.
• Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentation
  19 Mar 2020 2 repositories listed Syntology ran 0 of 13 samples · 13 unverified
  In addition, we address a key challenge in this multi-task setup, i.
• A Fast and Accurate One-Stage Approach to Visual Grounding
  18 Aug 2019 2 repositories listed Syntology ran 1 of 19 samples · 18 unverified
  We propose a simple, fast, and accurate one-stage approach to visual grounding, inspired by the following insight.
• Generating Easy-to-Understand Referring Expressions for Target Identifications
  29 Nov 2018 2 repositories listed
  Moreover, we regard that sentences that are easily understood are those that are comprehended correctly and quickly by humans.
• A Joint Speaker-Listener-Reinforcer Model for Referring Expressions
  30 Dec 2016 2 repositories listed
  The speaker generates referring expressions, the listener comprehends referring expressions, and the reinforcer introduces a reward function to guide sampling of more discriminative expressions.
• Detecting Referring Expressions in Visually Grounded Dialogue with Autoregressive Language Models
  26 Jun 2025 1 repository listed
  In this paper, we explore the use of a text-only, autoregressive language modeling approach for the extraction of referring expressions from visually grounded dialogue.
• TextRegion: Text-Aligned Region Tokens from Frozen Image-Text Models
  29 May 2025 1 repository listed
  Image-text models excel at image-level tasks but struggle with detailed visual understanding.

Syntology lines on 21 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.