Browse State-of-the-Art › Visual Entailment
Visual Entailment
33 papers with code · 3 benchmarks · 3 datasets archive 2025-07-28
Visual Entailment (VE) - is a task consisting of image-sentence pairs whereby a premise is defined by an image, rather than a natural language sentence as in traditional Textual Entailment tasks. The goal is to predict whether the image semantically entails the text.
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
3 leaderboard tables shown for this task, 3 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
3 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 56 · with a code link: 33 · where Syntology ran a sample: 12 (11 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (12 of 56 tagged: 11 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 56 of 56, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
25 Sep 2019 7 repositories listed Syntology community repositories only · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 2 pointer-only (licence)Different from previous work that applies joint random masking to both modalities, we use conditional masking on pre-training tasks (i. e., masked language/region modeling is conditioned on full observation of…
-
4 May 2022 6 repositories listed Syntology 10 ran (of which 5 constructed an object rather than computing a result; 9 with no instrument failure: 2 honoured, 0 violated, 7 with no contract checked; 1 where Syntology's instrument failed) · 7 unverified (of 17 harvested samples) · 2 pointer-only (licence)We apply a contrastive loss between unimodal image and text embeddings, in addition to a captioning loss on the multimodal decoder outputs which predicts text tokens autoregressively.
-
30 Apr 2022 4 repositories listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 1 violated, 4 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified (of 10 harvested samples) · 6 pointer-only (licence)Spatial relations are a basic part of human cognition.
-
7 Feb 2022 4 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)In this work, we pursue a unified paradigm for multimodal pretraining to break the scaffolds of complex task/modality-specific customization.
-
16 Dec 2021 4 repositories listedWe propose a cross-modal attention distillation framework to train a dual-encoder model for vision-language understanding tasks, such as visual reasoning and visual question answering.
-
13 Jul 2021 4 repositories listed Syntology official: harvested, nothing ran · 7 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 6 where Syntology's instrument failed) · 3 unverified (of 10 harvested samples) · 9 pointer-only (licence)Most existing Vision-and-Language (V&L) models rely on pre-trained visual encoders, using a relatively small set of manually-annotated data (as compared to web-crawled data), to perceive the visual world.
-
15 Dec 2023 3 repositories listedThis work inaugurates a new domain in factual error correction for chart captions, presenting a novel evaluation mechanism, and demonstrating an effective approach to ensuring the factuality of generated chart captions.
-
7 Apr 2021 3 repositories listedAs region-based visual features usually represent parts of an image, it is challenging for existing vision-language models to fully understand the semantics from paired natural languages.
-
11 Jun 2024 2 repositories listedGrounded Multimodal Named Entity Recognition (GMNER) task aims to identify named entities, entity types and their corresponding visual regions.
-
15 Feb 2024 2 repositories listedGrounded Multimodal Named Entity Recognition (GMNER) is a nascent multimodal task that aims to identify named entities, entity types and their corresponding visual regions.
-
11 Jun 2020 2 repositories listed Syntology official (archive's flag): 12 ran · 12 ran (of which 5 constructed an object rather than computing a result; 9 with no instrument failure: 1 honoured, 0 violated, 8 with no contract checked; 3 where Syntology's instrument failed) · 8 unverified (of 20 harvested samples) · 8 pointer-only (licence)We present VILLA, the first known effort on large-scale adversarial training for vision-and-language (V+L) representation learning.
-
19 Dec 2024 1 repository listedAdditionally, we introduce a reward-driven update optimization method to further enhance the quality of updates generated by multimodal models.
-
2 May 2024 1 repository listedTo close this gap, we propose a new task framing the figurative meaning understanding problem as an explainable visual entailment task, where the model has to predict whether the image (premise) entails a caption…
-
14 Mar 2024 1 repository listedDespite the demonstrated parameter efficiency of prompt-based multimodal fusion methods, their limited adaptivity and expressiveness often result in suboptimal performance compared to other tuning approaches.
-
5 Mar 2024 1 repository listedVisual entailment (VE) is a multimodal reasoning task consisting of image-sentence pairs whereby a promise is defined by an image, and a hypothesis is described by a sentence.
-
17 Dec 2023 1 repository listedIn this paper, we present a novel modeling framework that recasts adapter tuning after attention as a graph message passing process on attention graphs, where the projected query and value features and attention matrix…
-
4 Dec 2023 1 repository listedQVix enables a wider exploration of visual scenes, improving the LVLMs' reasoning accuracy and depth in tasks such as visual question answering and visual entailment.
-
29 Jun 2023 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples)Our evaluation across three distinct tasks (image-text retrieval, visual entailment, and natural language visual reasoning) demonstrates that this approach outperforms the state-of-the-art multilingual vision-language…
-
24 May 2023 1 repository listedWe propose to solve the task through the collaboration between Large Language Models (LLMs) and Diffusion Models: Instruct GPT-3 (davinci-002) with Chain-of-Thought prompting generates text that represents a visual…
-
15 Dec 2022 1 repository listed Syntology 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)Multimodal image-text models have shown remarkable performance in the past few years.
-
8 Dec 2022 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 1 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 8 harvested samples) · 6 pointer-only (licence)Natural language explanations promise to offer intuitively understandable explanations of a neural network's decision process in complex vision-language tasks, as pursued in recent VL-NLE models.
-
17 Nov 2022 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 6 harvested samples) · 1 pointer-only (licence)We produce models using only text training data on four representative tasks: image captioning, visual entailment, visual question answering and visual news captioning, and evaluate them on standard benchmarks using…
-
11 Oct 2022 1 repository listedMultimodal semantic understanding often has to deal with uncertainty, which means the obtained messages tend to refer to multiple targets.
-
29 Aug 2022 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)Vision and Language Pretraining has become the prevalent approach for tackling multimodal downstream tasks.
-
4 Aug 2022 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)Prompt tuning has become a new paradigm for model tuning and it has demonstrated success in natural language pretraining and even vision pretraining.
-
23 Jul 2022 1 repository listedCSI), a relation inferrer, and a Lexical Constraint-aware Generator (arr.
-
16 Jun 2022 1 repository listed Syntology official: harvested, nothing ran · 0 ran · 2 unverified (of 2 harvested samples)Data augmentation is a necessity to enhance data efficiency in deep learning.
-
29 Mar 2022 1 repository listedIn this paper, we propose an extension of this task, where the goal is to predict the logical relationship of fine-grained knowledge elements within a piece of text to an image.
-
9 Mar 2022 1 repository listedCurrent NLE models explain the decision-making process of a vision or vision-language model (a. k. a., task model), e. g., a VQA model, via a language model (a. k. a., explanation model), e. g., GPT.
-
1 Aug 2021 1 repository listedBesides, they only explore the interaction between image and question, ignoring the semantics of candidate answers.
-
8 Jun 2021 1 repository listed
-
20 Jan 2019 1 repository listed
-
26 Nov 2018 1 repository listed
-
VolDoGer: LLM-assisted Datasets for Domain Generalization in Vision-Language Tasks29 Jul 2024 0 repositories listed
-
ArcSin: Adaptive ranged cosine Similarity injected noise for Language-Driven Visual Tasks27 Feb 2024 0 repositories listed
-
Lightweight In-Context Tuning for Multimodal Unified Models8 Oct 2023 0 repositories listed
-
"Let's not Quote out of Context": Unified Vision-Language Pretraining for Context Assisted Image Captioning1 Jun 2023 0 repositories listed
-
Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning10 Mar 2023 0 repositories listed
-
Few-shot Multimodal Multitask Multilingual Learning19 Feb 2023 0 repositories listed
-
Compound Tokens: Channel Fusion for Vision-Language Representation Learning2 Dec 2022 0 repositories listed
-
A survey on knowledge-enhanced multimodal learning19 Nov 2022 0 repositories listed
-
AlignVE: Visual Entailment Recognition Based on Alignment Relations16 Nov 2022 0 repositories listed
-
Pre-training image-language transformers for open-vocabulary tasks9 Sep 2022 0 repositories listed
-
Answer-Me: Multi-Task Open-Vocabulary Visual Question Answering2 May 2022 0 repositories listed
-
22 Apr 2022 0 repositories listed
-
CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailment14 Mar 2022 0 repositories listed
-
Unsupervised Vision-and-Language Pre-training via Retrieval-based Multi-Granular Alignment1 Mar 2022 0 repositories listed
-
CLIP-TD: CLIP Targeted Distillation for Vision-Language Tasks15 Jan 2022 0 repositories listed
-
Logically at Factify 2022: Multimodal Fact Verification16 Dec 2021 0 repositories listed
-
Unified Multimodal Pre-training and Prompt-based Tuning for Vision-Language Understanding and Generation10 Dec 2021 0 repositories listed
-
How Much Can CLIP Benefit Vision-and-Language Tasks?29 Sep 2021 0 repositories listed
-
Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-training25 Jun 2021 0 repositories listed
-
21 May 2021 0 repositories listed
-
Playing Lottery Tickets with Vision and Language23 Apr 2021 0 repositories listed
-
UNITER: Learning UNiversal Image-TExt Representations25 Sep 2019 0 repositories listed
-
Segment-Phrase Table for Semantic Segmentation, Visual Entailment and Paraphrasing27 Sep 2015 0 repositories listed
Syntology lines on 13 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections