Phrase Grounding

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

5 leaderboard tables shown for this task, 5 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

6 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

1 subtask in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 51 papers with code (88 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
  6 Jun 2016 10 repositories listed
  Approaches to multimodal pooling include element-wise product or sum, as well as concatenation of the visual and textual representations.
• MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding
  26 Apr 2021 5 repositories listed Syntology ran 6 of 11 samples · 5 unverified
  We also investigate the utility of our model as an object detector on a given label set when fine-tuned in a few-shot setting.
• Towards Visual Grounding: A Survey
  28 Dec 2024 4 repositories listed Syntology ran 9 of 15 samples · 6 unverified · 3 pointer-only (licence)
  Finally, we outline the challenges confronting visual grounding and propose valuable directions for future research, which may serve as inspiration for subsequent researchers.
• Revisiting Image-Language Networks for Open-ended Phrase Detection
  17 Nov 2018 3 repositories listed
  Most existing work that grounds natural language phrases in images starts with the assumption that the phrase in question is relevant to the image.
• Grounding of Textual Phrases in Images by Reconstruction
  12 Nov 2015 3 repositories listed
  We propose a novel approach which learns grounding by reconstructing a given phrase using an attention mechanism, which can be either latent or optimized directly.
• An Open and Comprehensive Pipeline for Unified Object Grounding and Detection
  4 Jan 2024 2 repositories listed Syntology ran 4 of 6 samples · 2 unverified
  Grounding-DINO is a state-of-the-art open-set detection model that tackles multiple vision tasks including Open-Vocabulary Detection (OVD), Phrase Grounding (PG), and Referring Expression Comprehension (REC).
• Kosmos-2: Grounding Multimodal Large Language Models to the World
  26 Jun 2023 2 repositories listed
  We introduce Kosmos-2, a Multimodal Large Language Model (MLLM), enabling new capabilities of perceiving object descriptions (e.
• Making the Most of Text Semantics to Improve Biomedical Vision--Language Processing
  21 Apr 2022 2 repositories listed Syntology ran 3 of 4 samples · 1 unverified
  We release a new dataset with locally-aligned phrase grounding annotations by radiologists to facilitate the study of complex semantic modelling in biomedical vision--language processing.
• Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models
  19 May 2015 2 repositories listed Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
  The Flickr30k dataset has become a standard benchmark for sentence-based image description.
• Disambiguating Reference in Visually Grounded Dialogues through Joint Modeling of Textual and Multimodal Semantic Structures
  16 May 2025 1 repository listed Syntology ran 2 of 6 samples · 4 unverified
  Phrase grounding between images and their captions is a well-established task.
• Anatomical grounding pre-training for medical phrase grounding
  23 Feb 2025 1 repository listed
  Medical Phrase Grounding (MPG) maps radiological findings described in medical reports to specific regions in medical images.
• VICCA: Visual Interpretation and Comprehension of Chest X-ray Anomalies in Generated Report Without Human Feedback
  29 Jan 2025 1 repository listed
  As artificial intelligence (AI) becomes increasingly central to healthcare, the demand for explainable and trustworthy models is paramount.
• Context-Infused Visual Grounding for Art
  16 Oct 2024 1 repository listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  Many artwork collections contain textual attributes that provide rich and contextualised descriptions of artworks.
• Transformer with Controlled Attention for Synchronous Motion Captioning
  13 Sep 2024 1 repository listed
  In this paper, we address a challenging task, synchronous motion captioning, that aim to generate a language description synchronized with human motion sequences.
• A Lightweight Modular Framework for Low-Cost Open-Vocabulary Object Detection Training
  20 Aug 2024 1 repository listed
  The core of our approach involves freezing the MDETR backbone and training only the Universal Projection module (UP), which bridges vision and language representations.
• Empathic Grounding: Explorations using Multimodal Interaction and Large Language Models with Conversational Agents
  1 Jul 2024 1 repository listed
  We introduce the concept of "empathic grounding" in conversational agents as an extension of Clark's conceptualization of grounding in conversation in which the grounding criterion includes listener empathy for the…
• Zero-Shot Medical Phrase Grounding with Off-the-shelf Diffusion Models
  19 Apr 2024 1 repository listed Syntology ran 7 of 9 samples · 2 unverified
  In addition, we aim to perform this task in a zero-shot manner, i.
• Griffon v2: Advancing Multimodal Perception with High-Resolution Scaling and Visual-Language Co-Referring
  14 Mar 2024 1 repository listed
  Large Vision Language Models have achieved fine-grained object perception, but the limitation of image resolution remains a significant obstacle to surpass the performance of task-specific experts in complex and dense…
• PG-Video-LLaVA: Pixel Grounding Large Video-Language Models
  22 Nov 2023 1 repository listed Syntology ran 1 of 5 samples · 4 unverified · 5 pointer-only (licence)
  Extending image-based Large Multimodal Models (LMMs) to videos is challenging due to the inherent complexity of video data.
• Augment the Pairs: Semantics-Preserving Image-Caption Pair Augmentation for Grounding-Based Vision and Language Models
  5 Nov 2023 1 repository listed
  While we demonstrate our data augmentation method with MDETR framework, the proposed approach is applicable to common grounding-based vision and language tasks with other frameworks.
• Localizing Active Objects from Egocentric Vision with Symbolic World Knowledge
  23 Oct 2023 1 repository listed
  The ability to actively ground task instructions from an egocentric view is crucial for AI agents to accomplish tasks or assist humans virtually.
• Enhancing Representation in Radiography-Reports Foundation Model: A Granular Alignment Algorithm Using Masked Contrastive Learning
  12 Sep 2023 1 repository listed
  It designs a correlation weighting mechanism to adjust the correlation between masked chest X-ray image patches and their corresponding reports, thereby enhancing the model's representation learning capabilities.
• Box-based Refinement for Weakly Supervised and Unsupervised Localization Tasks
  7 Sep 2023 1 repository listed Syntology ran 6 of 11 samples · 5 unverified · 11 pointer-only (licence)
  It has been established that training a box-based detector network can enhance the localization performance of weakly supervised and unsupervised methods.
• A Joint Study of Phrase Grounding and Task Performance in Vision and Language Models
  6 Sep 2023 1 repository listed Syntology ran 10 of 12 samples · 2 unverified · 12 pointer-only (licence)
  Key to tasks that require reasoning about natural language in visual contexts is grounding words and phrases to image regions.
• A Survey on Interpretable Cross-modal Reasoning
  5 Sep 2023 1 repository listed
  In recent years, cross-modal reasoning (CMR), the process of understanding and reasoning across different modalities, has emerged as a pivotal area with applications spanning from multimedia analysis to healthcare…
• Trade-offs in Fine-tuned Diffusion Models Between Accuracy and Interpretability
  31 Mar 2023 1 repository listed
  Recent advancements in diffusion models have significantly impacted the trajectory of generative machine learning research, with many adopting the strategy of fine-tuning pre-trained models using domain-specific…
• Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing
  11 Jan 2023 1 repository listed
  Prior work in biomedical VLP has mostly relied on the alignment of single image and report pairs even though clinical notes commonly refer to prior images.
• Similarity Maps for Self-Training Weakly-Supervised Phrase Grounding
  1 Jan 2023 1 repository listed
  A phrase grounding model receives an input image and a text phrase and outputs a suitable localization map.
• DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Grounding
  28 Nov 2022 1 repository listed
  As phrase extraction can be regarded as a $1$D text segmentation problem, we formulate PEG as a dual detection problem and propose a novel DQ-DETR model, which introduces dual queries to probe different features from…
• Extending Phrase Grounding with Pronouns in Visual Dialogues
  23 Oct 2022 1 repository listed Syntology ran 0 of 9 samples · 9 unverified
  First, we construct a dataset of phrase grounding with both noun phrases and pronouns to image regions.

Syntology lines on 12 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.