Papers › Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
Akira Fukui, Dong Huk Park, Daylen Yang, Anna Rohrbach, Trevor Darrell, Marcus Rohrbach
Modeling textual or visual information with vector representations trained from large language or visual datasets has been successfully explored in recent years. However, tasks such as visual question answering require combining these vector representations with each other. Approaches to multimodal pooling include element-wise product or sum, as well as concatenation of the visual and textual representations. We hypothesize that these methods are not as expressive as an outer product of the visual and textual vectors. As the outer product is typically infeasible due to its high dimensionality, we instead propose utilizing Multimodal Compact Bilinear pooling (MCB) to efficiently and expressively combine multimodal features. We extensively evaluate MCB on the visual question answering and grounding tasks. We consistently show the benefit of MCB over ablations without MCB. For visual question answering, we present an architecture which uses MCB twice, once for predicting attention over spatial features and again to combine the attended representation with the question representation. This model outperforms the state-of-the-art on the Visual7W dataset and the VQA challenge.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Phrase Grounding | Flickr30k Entities Test | MCB | R@1 | 48.69 | #12 of 18 | Archive leaderboard | report |
| Phrase Grounding | ReferIt | MCB | Accuracy | 28.91 | #3 of 3 | Archive leaderboard | report |
| Visual Question Answering (VQA) | COCO Visual Question Answering (VQA) real images 1.0 multiple choice | MCB 7 att. | Percentage correct | 70.1 | #1 of 10 | Archive leaderboard | report |
| Visual Question Answering (VQA) | COCO Visual Question Answering (VQA) real images 1.0 open ended | MCB 7 att. | Percentage correct | 66.5 | #1 of 14 | Archive leaderboard | report |
| Visual Question Answering (VQA) | VQA v1 test-dev | MCB (ResNet) | Accuracy | 64.2 | #3 of 7 | Archive leaderboard | report |
| Visual Question Answering (VQA) | VQA v2 test-dev | MCB | Accuracy | 64.7 | #45 of 56 | Archive leaderboard | report |
| Visual Question Answering (VQA) | Visual7W | MCB+Att. | Percentage correct | 62.2 | #4 of 4 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections