Papers › MAttNet: Modular Attention Network for Referring Expression Comprehension
MAttNet: Modular Attention Network for Referring Expression Comprehension
Licheng Yu, Zhe Lin, Xiaohui Shen, Jimei Yang, Xin Lu, Mohit Bansal, Tamara L. Berg
In this paper, we address referring expression comprehension: localizing an image region described by a natural language expression. While most recent work treats expressions as a single unit, we propose to decompose them into three modular components related to subject appearance, location, and relationship to other objects. This allows us to flexibly adapt to expressions containing different types of information in an end-to-end framework. In our model, which we call the Modular Attention Network (MAttNet), two types of attention are utilized: language-based attention that learns the module weights as well as the word/phrase attention that each module should focus on; and visual attention that allows the subject and relationship modules to focus on relevant image components. Module weights combine scores from all three modules dynamically to output an overall score. Experiments show that MAttNet outperforms previous state-of-art methods by a large margin on both bounding-box-level and pixel-level comprehension tasks. Demo and code are provided.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Generalized Referring Expression Segmentation | gRefCOCO | MattNet | cIoU | 47.51 | #13 of 13 | Archive leaderboard | report |
| Generalized Referring Expression Segmentation | gRefCOCO | MattNet | gIoU | 48.24 | #13 of 13 | Archive leaderboard | report |
| Referring Expression Segmentation | RefCOCO+ test B | MattNet | Overall IoU | 40.08 | #26 of 30 | Archive leaderboard | report |
| Referring Expression Segmentation | RefCOCO+ testA | MattNet | Overall IoU | 52.39 | #26 of 30 | Archive leaderboard | report |
| Referring Expression Segmentation | RefCOCO+ val | MattNet | Overall IoU | 46.67 | #30 of 33 | Archive leaderboard | report |
| Referring Expression Segmentation | RefCoCo val | MattNet | Overall IoU | 56.51 | #36 of 37 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections