Papers › VTQA: Visual Text Question Answering via Entity Alignment and Cross-Media Reasoning

VTQA: Visual Text Question Answering via Entity Alignment and Cross-Media Reasoning

5 Mar 2023CVPR 2024 1arXiv:2303.02635archive 2025-07-28

Kang Chen, Xiangqian Wu

The ideal form of Visual Question Answering requires understanding, grounding and reasoning in the joint space of vision and language and serves as a proxy for the AI task of scene understanding. However, most existing VQA benchmarks are limited to just picking the answer from a pre-defined set of options and lack attention to text. We present a new challenge with a dataset that contains 23,781 questions based on 10124 image-text pairs. Specifically, the task requires the model to align multimedia representations of the same entity to implement multi-hop reasoning between image and text and finally use natural language to answer the question. The aim of this challenge is to develop and benchmark models that are capable of multimedia entity alignment, multi-step reasoning and open-ended answer generation.

PaperPDFConference PDFCode

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

visual-text-QA/VTQA-Demo officialmentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Answer GenerationEntity AlignmentQuestion AnsweringScene UnderstandingVisual Question AnsweringVisual Question Answering (VQA)

Datasets

Introduced by this paper, per the archive.

VTQA

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

ALIGN

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections