Browse State-of-the-Art › Visual Commonsense Reasoning
Visual Commonsense Reasoning
33 papers with code · 7 benchmarks · 8 datasets archive 2025-07-28
Image source: Visual Commonsense Reasoning
Description from the archive archive 2025-07-28; Papers-with-Code links inside it are rewritten to this site.
Benchmarks archive 2025-07-28
7 leaderboard tables shown for this task, 7 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
8 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 65 · with a code link: 33 · where Syntology ran a sample: 14 (11 with a run with no instrument failure, 3 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (14 of 65 tagged: 11 with a run with no instrument failure, 3 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 65 of 65, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
6 Aug 2019 11 repositories listed Syntology 10 ran (of which 6 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 2 where Syntology's instrument failed) · 24 unverified (of 34 harvested samples) · 34 pointer-only (licence)We present ViLBERT (short for Vision-and-Language BERT), a model for learning task-agnostic joint representations of image content and natural language.
-
25 Sep 2019 7 repositories listed Syntology community repositories only · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 2 pointer-only (licence)Different from previous work that applies joint random masking to both modalities, we use conditional masking on pre-training tasks (i. e., masked language/region modeling is conditioned on full observation of…
-
1 Dec 2023 4 repositories listedFurthermore, we present ViP-Bench, a comprehensive benchmark to assess the capability of models in understanding visual prompts across multiple dimensions, enabling future research in this domain.
-
27 Nov 2018 4 repositories listed Syntology 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples)While this task is easy for humans, it is tremendously difficult for today's vision systems, requiring higher-order cognition and commonsense reasoning about the world.
-
7 Jul 2023 3 repositories listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 5 harvested samples) · 5 pointer-only (licence)Before sending to LLM, the reference is replaced by RoI features and interleaved with language embeddings as a sequence.
-
22 Aug 2019 3 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)We introduce a new pre-trainable generic representation for visual-linguistic tasks, called Visual-Linguistic BERT (VL-BERT for short).
-
18 Aug 2021 2 repositories listedNevertheless, there has not been an open-source codebase in support of training and deploying numerous neural network models for cross-modal analytics in a unified and modular fashion.
-
4 Feb 2021 2 repositories listed Syntology official (archive's flag): 2 ran · 2 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 10 unverified; every one of the 2 samples that ran constructed an object rather than computing a result (of 12 harvested samples)On 7 popular vision-and-language benchmarks, including visual question answering, referring expression comprehension, visual commonsense reasoning, most of which have been previously modeled as discriminative tasks, our…
-
11 Jun 2020 2 repositories listed Syntology official (archive's flag): 12 ran · 12 ran (of which 5 constructed an object rather than computing a result; 9 with no instrument failure: 1 honoured, 0 violated, 8 with no contract checked; 3 where Syntology's instrument failed) · 8 unverified (of 20 harvested samples) · 8 pointer-only (licence)We present VILLA, the first known effort on large-scale adversarial training for vision-and-language (V+L) representation learning.
-
8 Dec 2024 1 repository listedLarge multimodal models (LMMs) have shown remarkable performance in the visual commonsense reasoning (VCR) task, which aims to answer a multiple-choice question based on visual commonsense within an image.
-
19 Jun 2024 1 repository listedTo facilitate multimodal grounded language modeling, we employ a late-fusion layer that combines the projected visual features with the output of a pre-trained LLM conditioned on text only.
-
3 Jun 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)Recent advances in vision-language models (VLMs) have demonstrated the advantages of processing images at higher resolutions and utilizing multi-crop features to preserve native resolution details.
-
5 Sep 2023 1 repository listedIn recent years, cross-modal reasoning (CMR), the process of understanding and reasoning across different modalities, has emerged as a pivotal area with applications spanning from multimedia analysis to healthcare…
-
1 Jan 2023 1 repository listedLanguage models are capable of commonsense reasoning: while domain-specific models can learn from explicit knowledge (e. g. commonsense graphs [6], ethical norms [25]), and larger models like GPT-3 manifest broad…
-
8 Dec 2022 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 8 harvested samples)We leverage situation recognition annotations and the CLIP model to generate a large set of 500k candidate analogies.
-
20 Sep 2022 1 repository listed Syntology official (archive's flag): 2 ran · 4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples) · 2 pointer-only (licence)We further design language models to learn to generate lectures and explanations as the chain of thought (CoT) to mimic the multi-hop reasoning process when answering ScienceQA questions.
-
17 Aug 2022 1 repository listedBootstrapping from pre-trained language models has been proven to be an efficient approach for building vision-language models (VLM) for tasks such as image captioning or visual question answering.
-
23 May 2022 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples)We show that PEVL enables state-of-the-art performance of detector-free VLP models on position-sensitive tasks such as referring expression comprehension and phrase grounding, and also improves the performance on…
-
30 Mar 2022 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples)Breakthroughs in transformer-based models have revolutionized not only the NLP field, but also vision and multimodal systems.
-
14 Mar 2022 1 repository listedIn this work, we for the first time introduce an end-to-end video-language model, namely all-in-one Transformer, that embeds raw video and textual signals into joint representations using a unified backbone architecture.
-
25 Feb 2022 1 repository listedGiven that our framework is model-agnostic, we apply it to the existing popular baselines and validate its effectiveness on the benchmark dataset.
-
27 Oct 2021 1 repository listedTo overcome this limitation and take a solid step towards artificial general intelligence (AGI), we develop a foundation model pre-trained with huge multimodal data, which can be quickly adapted for various downstream…
-
14 Sep 2021 1 repository listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 9 harvested samples)Commonsense is defined as the knowledge that is shared by everyone.
-
6 Aug 2021 1 repository listedWhile image understanding on recognition-level has achieved remarkable advancements, reliable visual scene understanding requires comprehensive image understanding on recognition-level but also cognition-level, which…
-
4 Jul 2021 1 repository listedMoreover, the proposed model provides intuitive interpretation into visual commonsense reasoning.
-
4 Jun 2021 1 repository listed Syntology 9 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 0 where Syntology's instrument failed) · 5 unverified (of 14 harvested samples) · 2 pointer-only (licence)As humans, we understand events in the visual world contextually, performing multimodal reasoning across time to make inferences about the past, present, and future.
-
15 Oct 2020 1 repository listedNatural language rationales could provide intuitive, higher-level explanations that are easily understandable by humans, complementing the more broadly studied lower-level explanations based on gradients or attention…
-
1 Dec 2019 1 repository listedInspired by this idea, towards VCR, we propose a connective cognition network (CCN) to dynamically reorganize the visual neuron connectivity that is contextualized by the meaning of questions and answers.
-
1 Dec 2019 1 repository listedDespite impressive recent progress that has been reported on tasks that necessitate reasoning, such as visual question answering and visual dialog, models often exploit biases in datasets.
-
31 Oct 2019 1 repository listedDespite impressive recent progress that has been reported on tasks that necessitate reasoning, such as visual question answering and visual dialog, models often exploit biases in datasets.
-
25 Oct 2019 1 repository listed
-
14 Aug 2019 1 repository listed
-
25 May 2018 1 repository listed
-
Compositional Image-Text Matching and Retrieval by Grounding Entities4 May 2025 0 repositories listed
-
Generative Visual Commonsense Answering and Explaining with Generative Scene Graph Constructing15 Jan 2025 0 repositories listed
-
How Vision-Language Tasks Benefit from Large Pre-trained Models: A Survey11 Dec 2024 0 repositories listed
-
Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense?11 Jun 2024 0 repositories listed
-
ALGO: Object-Grounded Visual Commonsense Reasoning for Open-World Egocentric Action Recognition9 Jun 2024 0 repositories listed
-
Do Vision-Language Transformers Exhibit Visual Commonsense? An Empirical Study of VCR27 May 2024 0 repositories listed
-
EventLens: Leveraging Event-Aware Pretraining and Cross-modal Linking Enhances Visual Commonsense Reasoning22 Apr 2024 0 repositories listed
-
Improving Vision-and-Language Reasoning via Spatial Relations Modeling9 Nov 2023 0 repositories listed
-
ViCor: Bridging Visual Understanding and Commonsense Reasoning with Large Language Models9 Oct 2023 0 repositories listed
-
Discovering Novel Actions from Open World Egocentric Videos with Object-Grounded Visual Commonsense Reasoning26 May 2023 0 repositories listed
-
GRILL: Grounded Vision-language Pre-training via Aligning Text and Image Regions24 May 2023 0 repositories listed
-
CAVL: Learning Contrastive and Adaptive Representations of Vision and Language10 Apr 2023 0 repositories listed
-
Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images13 Mar 2023 0 repositories listed
-
Learning to Agree on Vision Attention for Visual Commonsense Reasoning4 Feb 2023 0 repositories listed
-
Multi-modal Large Language Model Enhanced Pseudo 3D Perception Framework for Visual Commonsense Reasoning30 Jan 2023 0 repositories listed
-
A survey on knowledge-enhanced multimodal learning19 Nov 2022 0 repositories listed
-
On Advances in Text Generation from Images Beyond Captioning: A Case Study in Self-Rationalization24 May 2022 0 repositories listed
-
Super-Prompting: Utilizing Model-Independent Contextual Data to Reduce Data Annotation Required in Visual Commonsense Tasks25 Apr 2022 0 repositories listed
-
22 Apr 2022 0 repositories listed
-
Attention Mechanism based Cognition-level Scene Understanding17 Apr 2022 0 repositories listed
-
CLIP-TD: CLIP Targeted Distillation for Vision-Language Tasks15 Jan 2022 0 repositories listed
-
7 Jan 2022 0 repositories listed
-
SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoning16 Dec 2021 0 repositories listed
-
Premise-based Multimodal Reasoning: Conditional Inference on Joint Textual and Visual Clues15 May 2021 0 repositories listed
-
Playing Lottery Tickets with Vision and Language23 Apr 2021 0 repositories listed
-
13 Dec 2020 0 repositories listed
-
To Root Artificial Intelligence Deeply in Basic Science for a New Generation of AI11 Sep 2020 0 repositories listed
-
VisualCOMET: Reasoning about the Dynamic Context of a Still Image22 Apr 2020 0 repositories listed
-
30 Mar 2020 0 repositories listed
-
Enforcing Reasoning in Visual Commonsense Reasoning21 Oct 2019 0 repositories listed
-
UNITER: Learning UNiversal Image-TExt Representations25 Sep 2019 0 repositories listed
-
16 Aug 2019 0 repositories listed
Syntology lines on 14 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections