Referring Expression Comprehension

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

12 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 98 papers with code (167 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Visual Instruction Tuning
  17 Apr 2023 13 repositories listed Syntology ran 16 of 51 samples · 35 unverified
  Instruction tuning large language models (LLMs) using machine-generated instruction-following data has improved zero-shot capabilities on new tasks, but the idea is less explored in the multimodal field.
• ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
  6 Aug 2019 11 repositories listed Syntology ran 10 of 34 samples · 24 unverified · 34 pointer-only (licence)
  We present ViLBERT (short for Vision-and-Language BERT), a model for learning task-agnostic joint representations of image content and natural language.
• Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection
  9 Mar 2023 10 repositories listed Syntology ran 2 of 5 samples · 3 unverified
  To effectively fuse language and vision modalities, we conceptually divide a closed-set detector into three phases and propose a tight fusion solution, which includes a feature enhancer, a language-guided query…
• Compositional Attention Networks for Machine Reasoning
  8 Mar 2018 10 repositories listed Syntology ran 1 of 7 samples · 6 unverified
  We present the MAC network, a novel fully differentiable neural network architecture, designed to facilitate explicit and expressive reasoning.
• Improved Baselines with Visual Instruction Tuning
  5 Oct 2023 9 repositories listed Syntology ran 6 of 9 samples · 3 unverified · 8 pointer-only (licence)
  Large multimodal models (LMM) have recently shown encouraging progress with visual instruction tuning.
• UNITER: UNiversal Image-TExt Representation Learning
  25 Sep 2019 7 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 2 pointer-only (licence)
  Different from previous work that applies joint random masking to both modalities, we use conditional masking on pre-training tasks (i.
• MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding
  26 Apr 2021 5 repositories listed Syntology ran 6 of 11 samples · 5 unverified
  We also investigate the utility of our model as an object detector on a given label set when fine-tuned in a few-shot setting.
• Towards Visual Grounding: A Survey
  28 Dec 2024 4 repositories listed Syntology ran 9 of 15 samples · 6 unverified · 3 pointer-only (licence)
  Finally, we outline the challenges confronting visual grounding and propose valuable directions for future research, which may serve as inspiration for subsequent researchers.
• Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V
  17 Oct 2023 4 repositories listed Syntology ran 0 of 1 samples · 1 unverified
  We present Set-of-Mark (SoM), a new visual prompting method, to unleash the visual grounding abilities of large multimodal models (LMMs), such as GPT-4V.
• OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework
  7 Feb 2022 4 repositories listed Syntology ran 1 of 1 samples · 0 unverified
  In this work, we pursue a unified paradigm for multimodal pretraining to break the scaffolds of complex task/modality-specific customization.
• SeqTR: A Simple yet Universal Network for Visual Grounding
  30 Mar 2022 3 repositories listed Syntology ran 1 of 4 samples · 3 unverified
  In this paper, we propose a simple yet universal network termed SeqTR for visual grounding tasks, e.
• VL-BERT: Pre-training of Generic Visual-Linguistic Representations
  22 Aug 2019 3 repositories listed Syntology ran 0 of 1 samples · 1 unverified
  We introduce a new pre-trainable generic representation for visual-linguistic tasks, called Visual-Linguistic BERT (VL-BERT for short).
• CLEVR-Ref+: Diagnosing Visual Reasoning with Referring Expressions
  3 Jan 2019 3 repositories listed Syntology ran 4 of 5 samples · 1 unverified · 5 pointer-only (licence)
  Yet there has been evidence that current benchmark datasets suffer from bias, and current state-of-the-art models cannot be easily evaluated on their intermediate reasoning process.
• Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models
  27 Mar 2024 2 repositories listed Syntology ran 7 of 8 samples · 1 unverified
  We try to narrow the gap by mining the potential of VLMs for better performance and any-to-any workflow from three aspects, i.
• LLMs as Bridges: Reformulating Grounded Multimodal Named Entity Recognition
  15 Feb 2024 2 repositories listed
  Grounded Multimodal Named Entity Recognition (GMNER) is a nascent multimodal task that aims to identify named entities, entity types and their corresponding visual regions.
• An Open and Comprehensive Pipeline for Unified Object Grounding and Detection
  4 Jan 2024 2 repositories listed Syntology ran 4 of 6 samples · 2 unverified
  Grounding-DINO is a state-of-the-art open-set detection model that tackles multiple vision tasks including Open-Vocabulary Detection (OVD), Phrase Grounding (PG), and Referring Expression Comprehension (REC).
• MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning
  14 Oct 2023 2 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 1 pointer-only (licence)
  Motivated by this, we target to build a unified interface for completing many vision-language tasks including image description, visual question answering, and visual grounding, among others.
• Kosmos-2: Grounding Multimodal Large Language Models to the World
  26 Jun 2023 2 repositories listed
  We introduce Kosmos-2, a Multimodal Large Language Model (MLLM), enabling new capabilities of perceiving object descriptions (e.
• ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities
  18 May 2023 2 repositories listed Syntology ran 2 of 7 samples · 5 unverified
  In this work, we explore a scalable way for building a general representation model toward unlimited modalities.
• ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehension
  12 Apr 2022 2 repositories listed Syntology ran 0 of 5 samples · 5 unverified
  Training a referring expression comprehension (ReC) model for a new visual domain requires collecting referring expressions, and potentially corresponding bounding boxes, for images in the domain.
• TransVG: End-to-End Visual Grounding with Transformers
  17 Apr 2021 2 repositories listed Syntology ran 6 of 20 samples · 14 unverified
  In this paper, we present a neat yet effective transformer-based framework for visual grounding, namely TransVG, to address the task of grounding a language query to the corresponding region onto an image.
• Unifying Vision-and-Language Tasks via Text Generation
  4 Feb 2021 2 repositories listed Syntology ran 2 of 12 samples · 10 unverified
  On 7 popular vision-and-language benchmarks, including visual question answering, referring expression comprehension, visual commonsense reasoning, most of which have been previously modeled as discriminative tasks, our…
• Large-Scale Adversarial Training for Vision-and-Language Representation Learning
  11 Jun 2020 2 repositories listed Syntology ran 10 of 20 samples · 10 unverified · 6 pointer-only (licence)
  We present VILLA, the first known effort on large-scale adversarial training for vision-and-language (V+L) representation learning.
• Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentation
  19 Mar 2020 2 repositories listed Syntology ran 0 of 13 samples · 13 unverified
  In addition, we address a key challenge in this multi-task setup, i.
• A Fast and Accurate One-Stage Approach to Visual Grounding
  18 Aug 2019 2 repositories listed Syntology ran 1 of 19 samples · 18 unverified
  We propose a simple, fast, and accurate one-stage approach to visual grounding, inspired by the following insight.
• A Joint Speaker-Listener-Reinforcer Model for Referring Expressions
  30 Dec 2016 2 repositories listed
  The speaker generates referring expressions, the listener comprehends referring expressions, and the reinforcer introduces a reward function to guide sampling of more discriminative expressions.
• TextRegion: Text-Aligned Region Tokens from Frozen Image-Text Models
  29 May 2025 1 repository listed
  Image-text models excel at image-level tasks but struggle with detailed visual understanding.
• VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model
  10 Apr 2025 1 repository listed
  Motivated by this observation, we investigate the extension of R1-style reinforcement learning to Vision-Language Models (VLMs), aiming to enhance their visual reasoning capabilities.
• New Dataset and Methods for Fine-Grained Compositional Referring Expression Comprehension via Specialist-MLLM Collaboration
  27 Feb 2025 1 repository listed
  To address fine-grained compositional REC, we propose novel methods based on a Specialist-MLLM collaboration framework, leveraging the complementary strengths of them: Specialist Models handle simpler tasks efficiently,…
• RefDrone: A Challenging Benchmark for Referring Expression Comprehension in Drone Scenes
  1 Feb 2025 1 repository listed
  NGDINO explicitly learns and utilizes the number of objects referred to in the expression.

Syntology lines on 23 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.