Papers › Dual Attention Networks for Multimodal Reasoning and Matching

Dual Attention Networks for Multimodal Reasoning and Matching

2 Nov 2016CVPR 2017 7arXiv:1611.00471archive 2025-07-28

Hyeonseob Nam, Jung-Woo Ha, Jeonghee Kim

We propose Dual Attention Networks (DANs) which jointly leverage visual and textual attention mechanisms to capture fine-grained interplay between vision and language. DANs attend to specific regions in images and words in text through multiple steps and gather essential information from both modalities. Based on this framework, we introduce two types of DANs for multimodal reasoning and matching, respectively. The reasoning model allows visual and textual attentions to steer each other during collaborative inference, which is useful for tasks such as Visual Question Answering (VQA). In addition, the matching model exploits the two attention mechanisms to estimate the similarity between images and sentences by focusing on their shared semantics. Our extensive experiments validate the effectiveness of DANs in combining vision and language, achieving the state-of-the-art performance on public benchmarks for VQA and image-text matching.

Paper PDF Conference PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

iammrhelo/pytorch-vqa-dan mentioned on GitHubpytorch report

JunweiLiang/DualAttentionNetwork tf report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Collaborative Inference Image-text matching Multimodal Reasoning Question Answering Text Matching Visual Question Answering Visual Question Answering (VQA)

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Image Retrieval	Flickr30K 1K test	DAN	R@1	39.4	#12 of 18	Archive leaderboard	report
Image Retrieval	Flickr30K 1K test	DAN	R@10	79.1	#12 of 18	Archive leaderboard	report
Image Retrieval	Flickr30K 1K test	DAN	R@5	69.2	#12 of 18	Archive leaderboard	report
Visual Question Answering (VQA)	VQA v1 test-dev	DAN (ResNet)	Accuracy	64.3	#2 of 7	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

1x1 Convolution Average Pooling Batch Normalization Bottleneck Residual Block Convolution Global Average Pooling Kaiming Initialization Max Pooling ReLU Residual Block Residual Connection

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections