Papers › A Cognitive Paradigm Approach to Probe the Perception-Reasoning Interface in VLMs
A Cognitive Paradigm Approach to Probe the Perception-Reasoning Interface in VLMs
Mohit Vaishnav, Tanel Tammet
A fundamental challenge in artificial intelligence involves understanding the cognitive mechanisms underlying visual reasoning in sophisticated models like Vision-Language Models (VLMs). How do these models integrate visual perception with abstract thought, especially when reasoning across multiple images or requiring fine-grained compositional understanding? Drawing inspiration from cognitive science, this paper introduces a structured evaluation framework using diverse visual reasoning tasks-Bongard Problems (BPs) and Winoground-to dissect the perception-reasoning interface in VLMs. We propose three distinct evaluation paradigms, mirroring human problem-solving strategies: Direct Visual Rule Learning (DVRL; holistic processing), Deductive Rule Learning (DRL; rule extraction and application), and Componential Analysis (CA; analytical decomposition via task-agnostic textual descriptions). These paradigms systematically vary cognitive load and probe processing stages. Notably, CA enables multi-image reasoning evaluation even for single-image architectures and isolates reasoning from perception by operating on textual descriptions. Applying this framework, we demonstrate that CA, leveraging powerful language models for reasoning over rich, independently generated descriptions, achieves new state-of-the-art (SOTA) performance on challenging benchmarks including Bongard-OpenWorld, Bongard-HOI, and Winoground. Ablation studies confirm reasoning improves significantly when perceptual challenges are mitigated, revealing a critical perception bottleneck. Our framework provides a valuable diagnostic tool and suggests that decoupling perception (via rich, task-agnostic description) from reasoning is a promising direction for robust and general visual intelligence.
In Syntology View this paper on Syntology: its repositories, every harvested function with whether it ran, its licence and the call to fetch it.
Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Few-Shot Image Classification | Bongard-HOI | GPT-4o + CA | Avg. Accuracy | 77.3 | #2 of 9 | Archive leaderboard | report |
| Few-Shot Image Classification | Bongard-HOI | Gemini 2.0 + CA | Avg. Accuracy | 74.5 | #4 of 9 | Archive leaderboard | report |
| Visual Reasoning | Bongard-OpenWorld | Gemini-2.0 + CA | 2-Class Accuracy | 93.6 | #1 of 9 | Archive leaderboard | report |
| Visual Reasoning | Bongard-OpenWorld | GPT-4o + CA | 2-Class Accuracy | 92.8 | #2 of 9 | Archive leaderboard | report |
| Visual Reasoning | Winoground | GPT-4o + CA | Group Score | 52 | #1 of 114 | Archive leaderboard | report |
| Visual Reasoning | Winoground | GPT-4o + CA | Image Score | 58.5 | #1 of 114 | Archive leaderboard | report |
| Visual Reasoning | Winoground | GPT-4o + CA | Text Score | 75.5 | #1 of 114 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections