Papers › Visual relationship detection with deep structural ranking
Visual relationship detection with deep structural ranking
Kongming Liang, Yuhong Guo, Hong Chang, Xilin Chen
Visual relationship detection aims to describe the interactions between pairs of objects. Different from individual object learning tasks, the number of possible relationships is much larger, which makes it hard to explore only based on the visual appearance of objects. In addition, due to the limited human effort, the annotations for visual relationships are usually incomplete which increases the difficulty of model training and evaluation. In this paper, we propose a novel framework, called Deep Structural Ranking, for visual relationship detection. To complement the representation ability of visual appearance, we integrate multiple cues for predicting the relationships contained in an input image. Moreover, we design a new ranking objective function by enforcing the annotated relationships to have higher relevance scores. Unlike previous works, our proposed method can both facilitate the co-occurrence of relationships and mitigate the incompleteness problem. Experimental results show that our proposed method outperforms the state-of-the-art on the two widely used datasets. We also demonstrate its superiority in detecting zero-shot relationships.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Visual Relationship Detection | VRD Predicate Detection | vrd-dsr | R@100 | 93.18 | #2 of 7 | Archive leaderboard | report |
| Visual Relationship Detection | VRD Predicate Detection | vrd-dsr | R@50 | 86.01 | #2 of 7 | Archive leaderboard | report |
| Visual Relationship Detection | VRD Relationship Detection | vrd-dsr | R@100 | 23.29 | #2 of 8 | Archive leaderboard | report |
| Visual Relationship Detection | VRD Relationship Detection | vrd-dsr | R@50 | 19.03 | #2 of 8 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections