Referring expression generation

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

2 leaderboard tables shown for this task, 2 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

4 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

26 shown of 26 papers with code (84 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Visual Instruction Tuning
  17 Apr 2023 13 repositories listed Syntology ran 16 of 51 samples · 35 unverified
  Instruction tuning large language models (LLMs) using machine-generated instruction-following data has improved zero-shot capabilities on new tasks, but the idea is less explored in the multimodal field.
• Improved Baselines with Visual Instruction Tuning
  5 Oct 2023 9 repositories listed Syntology ran 6 of 9 samples · 3 unverified · 8 pointer-only (licence)
  Large multimodal models (LMM) have recently shown encouraging progress with visual instruction tuning.
• Modeling Context in Referring Expressions
  31 Jul 2016 4 repositories listed Syntology ran 1 of 4 samples · 3 unverified
  Humans refer to objects in their environments all the time, especially in dialogue with other people.
• Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models
  27 Mar 2024 2 repositories listed Syntology ran 7 of 8 samples · 1 unverified
  We try to narrow the gap by mining the potential of VLMs for better performance and any-to-any workflow from three aspects, i.
• MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning
  14 Oct 2023 2 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 1 pointer-only (licence)
  Motivated by this, we target to build a unified interface for completing many vision-language tasks including image description, visual question answering, and visual grounding, among others.
• Kosmos-2: Grounding Multimodal Large Language Models to the World
  26 Jun 2023 2 repositories listed
  We introduce Kosmos-2, a Multimodal Large Language Model (MLLM), enabling new capabilities of perceiving object descriptions (e.
• Vision-Language Models Are Not Pragmatically Competent in Referring Expression Generation
  22 Apr 2025 1 repository listed
  Referring Expression Generation (REG) is a core task for evaluating the pragmatic competence of vision-language systems, requiring not only accurate semantic grounding but also adherence to principles of cooperative…
• Frontiers in Intelligent Colonoscopy
  22 Oct 2024 1 repository listed Syntology ran 0 of 6 samples · 6 unverified
  Colonoscopy is currently one of the most sensitive screening methods for colorectal cancer.
• Grounding Language in Multi-Perspective Referential Communication
  4 Oct 2024 1 repository listed Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
  We introduce a task and dataset for referring expression generation and comprehension in multi-agent embodied environments.
• Uni-Med: A Unified Medical Generalist Foundation Model For Multi-Task Learning Via Connector-MoE
  26 Sep 2024 1 repository listed
  To mitigate the tug-of-war problem of multi-modal multi-task optimization in MLLMs, recent advances primarily focus on improving the LLM components, while neglecting the connector that bridges the gap between modalities.
• Referring Expression Generation in Visually Grounded Dialogue with Discourse-aware Comprehension Guiding
  9 Sep 2024 1 repository listed
  Second, we propose the use of discourse-aware comprehension guiding as part of a generate-and-rerank strategy through which candidate REs generated with our REG model are reranked based on their discourse-dependent…
• Resilience through Scene Context in Visual Referring Expression Generation
  18 Apr 2024 1 repository listed
  Scene context is well known to facilitate humans' perception of visible objects.
• Elysium: Exploring Object-level Perception in Videos via MLLM
  25 Mar 2024 1 repository listed Syntology ran 7 of 8 samples · 1 unverified · 8 pointer-only (licence)
  Multi-modal Large Language Models (MLLMs) have demonstrated their ability to perceive objects in still images, but their application in video-related tasks, such as object tracking, remains understudied.
• Multi-modal Instruction Tuned LLMs with Fine-grained Visual Perception
  5 Mar 2024 1 repository listed Syntology ran 4 of 7 samples · 3 unverified
  Multimodal Large Language Model (MLLMs) leverages Large Language Models as a cognitive framework for diverse visual-language tasks.
• Efficient Multimodal Learning from Data-centric Perspective
  18 Feb 2024 1 repository listed Syntology ran 4 of 7 samples · 3 unverified
  Multimodal Large Language Models (MLLMs) have demonstrated notable capabilities in general visual understanding and reasoning tasks.
• MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devices
  28 Dec 2023 1 repository listed
  We present MobileVLM, a competent multimodal vision language model (MMVLM) targeted to run on mobile devices.
• GLaMM: Pixel Grounding Large Multimodal Model
  6 Nov 2023 1 repository listed Syntology ran 1 of 1 samples · 0 unverified
  In this work, we present Grounding LMM (GLaMM), the first model that can generate natural language responses seamlessly intertwined with corresponding object segmentation masks.
• Collecting Visually-Grounded Dialogue with A Game Of Sorts
  10 Sep 2023 1 repository listed
  We address these concerns by introducing a collaborative image ranking task, a grounded agreement game we call "A Game Of Sorts".
• Whether you can locate or not? Interactive Referring Expression Generation
  19 Aug 2023 1 repository listed
  Referring Expression Generation (REG) aims to generate unambiguous Referring Expressions (REs) for objects in a visual scene, with a dual task of Referring Expression Comprehension (REC) to locate the referred object.
• LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day
  1 Jun 2023 1 repository listed
  In this paper, we propose a cost-efficient approach for training a vision-language conversational assistant that can answer open-ended research questions of biomedical images.
• Pento-DIARef: A Diagnostic Dataset for Learning the Incremental Algorithm for Referring Expression Generation from Examples
  24 May 2023 1 repository listed
  NLP tasks are typically defined extensionally through datasets containing example instantiations (e.
• Enriching the E2E dataset
  1 Aug 2021 1 repository listed
  This study introduces an enriched version of the E2E dataset, one of the most popular language resources for data-to-text NLG.
• Improving Quality and Efficiency in Plan-based Neural Data-to-Text Generation
  22 Sep 2019 1 repository listed
  We follow the step-by-step approach to neural data-to-text generation we proposed in Moryossef et al (2019), in which the generation process is divided into a text-planning stage followed by a plan-realization stage.
• Referring Expression Generation Using Entity Profiles
  4 Sep 2019 1 repository listed
  Referring Expression Generation (REG) is the task of generating contextually appropriate references to entities.
• Enriching the WebNLG corpus
  1 Nov 2018 1 repository listed
  This paper describes the enrichment of WebNLG corpus (Gardent et al., 2017a, b), with the aim to further extend its usefulness as a resource for evaluating common NLG tasks, including Discourse Ordering, Lexicalization…
• NeuralREG: An end-to-end approach to referring expression generation
  21 May 2018 1 repository listed
  Traditionally, Referring Expression Generation (REG) models first decide on the form and then on the content of references to discourse entities in text, typically relying on features such as salience and grammatical…

Syntology lines on 11 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.