Browse State-of-the-Art › Visual Relationship Detection
Visual Relationship Detection
37 papers with code · 5 benchmarks · 5 datasets archive 2025-07-28
Visual relationship detection (VRD) is one newly developed computer vision task aiming to recognize relations or interactions between objects in an image. It is a further learning task after object recognition and is essential for fully understanding images, even the visual world.
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
5 leaderboard tables shown for this task, 5 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| VRD Relationship Detection (8 rows) | Yu et. al [[Yu et al.2017a]] | Visual Relationship Detection with Internal and External... | — | — | Compare |
| VRD Predicate Detection (7 rows) | Yu et. al [[Yu et al.2017a]] | Visual Relationship Detection with Internal and External... | — | — | Compare |
| VRD Phrase Detection (7 rows) | Yu et. al [[Yu et al.2017a]] | Visual Relationship Detection with Internal and External... | — | — | Compare |
| Visual Genome (1 row) | PEVL | PEVL: Position-enhanced Pre-training and Prompt Tuning for... | code | Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified | Compare |
| VRD (1 row) | Ours - v | Improving Visual Relation Detection using Depth Maps | code | — | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
5 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
2 subtasks in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 82 · with a code link: 37 · where Syntology ran a sample: 9 (9 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (9 of 82 tagged: 9 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 82 of 82, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
25 Mar 2020 3 repositories listed Syntology official (archive's flag): 2 ran · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 2 honoured, 1 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples)We use these benchmarks to study the performance of several state-of-the-art long-tail models on the LTVRR setup.
-
7 Mar 2019 3 repositories listed Syntology 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 6 harvested samples)The first, Entity Instance Confusion, occurs when the model confuses multiple instances of the same type of entity (e. g. multiple cups).
-
20 Nov 2021 2 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)Furthermore, background knowledge represented by RWFNs can be used to alleviate the incompleteness of training sets even though the space complexity of RWFNs is much smaller than LTNs (1:27 ratio).
-
26 Jul 2021 2 repositories listed Syntology official (archive's flag): 8 ran · 13 ran (of which 4 constructed an object rather than computing a result; 12 with no instrument failure: 0 honoured, 0 violated, 12 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 14 harvested samples) · 6 pointer-only (licence)Compared to the task of scene graph generation from images, it is more challenging because of the dynamic relationships between objects and the temporal dependencies between frames allowing for a richer semantic…
-
21 Nov 2020 2 repositories listedDespite being widely used as a performance measure for visual detection tasks, Average Precision (AP) is limited in (i) reflecting localisation quality, (ii) interpretability and (iii) robustness to the design choices…
-
1 Oct 2019 2 repositories listedThis requires the detection of visual relationships: triples (subject, relation, object) describing a semantic relation between a subject and an object.
-
10 May 2025 1 repository listedIn this paper, we propose Mutual EnhancemenT of Objects and Relationships (METOR), a query-based unified framework to jointly model and mutually enhance object detection and relationship classification in…
-
26 Mar 2024 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 3 pointer-only (licence)Groupwise Query Specialization trains a specialized query by dividing queries and relations into disjoint groups and directing a query in a specific query group solely toward relations in the corresponding relation…
-
6 Mar 2024 1 repository listedSecondly, classifiers trained by a single, monolithic neural network often lack stability and generalization.
-
8 Nov 2023 1 repository listedWe present a novel self-supervised approach for representation learning, particularly for the task of Visual Relationship Detection (VRD).
-
13 Sep 2023 1 repository listedIn addition to spatial relations, we also propose 50K visual depictions across 10 temporal relations, consisting of videos depicting event/time-point interactions.
-
16 Mar 2023 1 repository listedTo address this challenge, we propose UniVRD, a novel bottom-up method for Unified Visual Relationship Detection by leveraging vision and language models (VLMs).
-
23 Aug 2022 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples)In this work, (1) we propose a novel three-decoder architecture as the infrastructure for focused attention; 2) we use the generalized intersection box prediction task to effectively guide our model to focus on…
-
8 Aug 2022 1 repository listed Syntology official (archive's flag): 2 ran · 3 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples) · 3 pointer-only (licence)Visual relationship detection aims to detect the interactions between objects in an image; however, this task suffers from combinatorial explosion due to the variety of objects and interactions.
-
23 May 2022 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples)We show that PEVL enables state-of-the-art performance of detector-free VLP models on position-sensitive tasks such as referring expression comprehension and phrase grounding, and also improves the performance on…
-
27 Jul 2021 1 repository listedThere is a surge of interest in image scene graph generation (object, attribute and relationship detection) due to the need of building fine-grained image understanding models that go beyond object detection.
-
5 Jul 2021 1 repository listedGiven input images, scene graph generation (SGG) aims to produce comprehensive, graphical representations describing visual relationships among salient objects.
-
26 Apr 2021 1 repository listedTo enable progress on this task, we create a new dataset consisting of 220k human-annotated 2.
-
24 Apr 2021 1 repository listedThis paper shows that modeling an effective message-passing flow through an attention mechanism can be critical to tackling the compositionality and long-tail challenges in VRR.
-
1 Jan 2021 1 repository listedScene Graph Generators (SGGs) are models that, given an image, build a directed graph where each edge represents a predicted subject predicate object triplet.
-
17 Dec 2020 1 repository listedAnalyzing the interactions between humans and objects from a video includes identification of the relationships between humans and the objects present in the video.
-
17 Oct 2020 1 repository listedWe want to analyze visually, to what extend team members and external developers contribute to open-source projects.
-
10 Sep 2020 1 repository listedVisual relationship detection aims to reason over relationships among salient objects in images, which has drawn increasing attention over the past few years.
-
16 Jun 2020 1 repository listedVisual relationship detection is fundamental for holistic image understanding.
-
16 Mar 2020 1 repository listedTo address this problem, we propose an attention based model, namely AVR, to achieve salient visual relationships based on both local and global context of the relationships.
-
14 Jan 2020 1 repository listedDetected objects, their labels and the discovered relations can be used to construct a scene graph which provides an abstract semantic interpretation of an image.
-
2 Nov 2019 1 repository listedVisual relationship detection, as a challenging task used to find and distinguish the interactions between object pairs in one image, has received much attention recently.
-
2 May 2019 1 repository listedWe argue that depth maps can additionally provide valuable information on object relations, e. g. helping to detect not only spatial relations, such as standing behind, but also non-spatial relations, such as holding.
-
16 Apr 2019 1 repository listedVisual relationship detection is an intermediate image understanding task that detects two objects and classifies a predicate that explains the relationship between two objects in an image.
-
15 Feb 2019 1 repository listedDetecting visual relationships, i. e.
-
BLOCK: Bilinear Superdiagonal Fusion for Visual Question Answering and Visual Relationship Detection31 Jan 2019 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)
-
2 Nov 2018 1 repository listed
-
29 Jun 2018 1 repository listed
-
27 Apr 2018 1 repository listed
-
1 Oct 2017 1 repository listed
-
8 Mar 2017 1 repository listed
-
21 Nov 2016 1 repository listed
-
End-to-end Open-vocabulary Video Visual Relationship Detection using Multi-modal Prompting19 Sep 2024 0 repositories listed
-
Scene-Graph ViT: End-to-End Open-Vocabulary Visual Relationship Detection21 Mar 2024 0 repositories listed
-
RelVAE: Generative Pretraining for few-shot Visual Relationship Detection27 Nov 2023 0 repositories listed
-
NeSy4VRD: A Multifaceted Resource for Neurosymbolic AI Research using Knowledge Graphs in Visual Relationship Detection22 May 2023 0 repositories listed
-
Image Semantic Relation Generation19 Oct 2022 0 repositories listed
-
Learning Structured Representations of Visual Scenes9 Jul 2022 0 repositories listed
-
VReBERT: A Simple and Flexible Transformer for Visual Relationship Detection18 Jun 2022 0 repositories listed
-
Scene Graph Generation: A Comprehensive Survey3 Jan 2022 0 repositories listed
-
A Probabilistic Graphical Model Based on Neural-Symbolic Reasoning for Visual Relationship Detection1 Jan 2022 0 repositories listed
-
BGT-Net: Bidirectional GRU Transformer Network for Scene Graph Generation11 Sep 2021 0 repositories listed
-
Visual Relationship Detection Using Part-and-Sum Transformers with Composite Queries5 May 2021 0 repositories listed
-
A Comprehensive Survey of Scene Graphs: Generation and Application17 Mar 2021 0 repositories listed
-
Towards Overcoming False Positives in Visual Relationship Detection23 Dec 2020 0 repositories listed
-
Constructing a Visual Relationship Authenticity Dataset11 Oct 2020 0 repositories listed
-
Bounding-box Channels for Visual Relationship Detection1 Aug 2020 0 repositories listed
-
Hierarchical Graph Attention Network for Visual Relationship Detection1 Jun 2020 0 repositories listed
-
Fixed-size Objects Encoding for Visual Relationship Detection29 May 2020 0 repositories listed
-
Visual Relationship Detection using Scene Graphs: A Survey16 May 2020 0 repositories listed
-
CPARR: Category-based Proposal Analysis for Referring Relationships17 Apr 2020 0 repositories listed
-
Deep Adaptive Semantic Logic (DASL): Compiling Declarative Knowledge into Deep Neural Networks16 Mar 2020 0 repositories listed
-
ReLaText: Exploiting Visual Relationships for Arbitrary-Shaped Scene Text Detection with Graph Convolutional Networks16 Mar 2020 0 repositories listed
-
Visual Relationship Detection with Low Rank Non-Negative Tensor Decomposition22 Nov 2019 0 repositories listed
-
Leveraging Auxiliary Text for Deep Recognition of Unseen Visual Relationships27 Oct 2019 0 repositories listed
-
Deep Contextual Attention for Human-Object Interaction Detection17 Oct 2019 0 repositories listed
-
Contextual Translation Embedding for Visual Relationship Detection and Scene Graph Generation28 May 2019 0 repositories listed
-
On Exploring Undetermined Relationships for Visual Relationship Detection5 May 2019 0 repositories listed
-
Target-Tailored Source-Transformation for Scene Graph Generation3 Apr 2019 0 repositories listed
-
Scene Graph Generation with External Knowledge and Image Reconstruction1 Apr 2019 0 repositories listed
-
Optimising the Input Image to Improve Visual Relationship Detection26 Mar 2019 0 repositories listed
-
On Class Imbalance and Background Filtering in Visual Relationship Detection20 Mar 2019 0 repositories listed
-
Visual Semantic Information Pursuit: A Survey13 Mar 2019 0 repositories listed
-
An Interpretable Model for Scene Graph Generation21 Nov 2018 0 repositories listed
-
Introduction to the 1st Place Winning Model of OpenImages Relationship Detection Challenge1 Nov 2018 0 repositories listed
-
A Problem Reduction Approach for Visual Relationships Detection26 Sep 2018 0 repositories listed
-
Context-Dependent Diffusion Network for Visual Relationship Detection11 Sep 2018 0 repositories listed
-
Improving Visual Relationship Detection using Semantic Modeling of Scene Descriptions1 Sep 2018 0 repositories listed
-
Improving Information Extraction from Images with Learned Semantic Models27 Aug 2018 0 repositories listed
-
Tensorize, Factorize and Regularize: Robust Visual Relationship Learning1 Jun 2018 0 repositories listed
-
Visual Relationship Detection Based on Guided Proposals and Semantic Knowledge Distillation28 May 2018 0 repositories listed
-
Attend and Interact: Higher-Order Object Interactions for Video Understanding16 Nov 2017 0 repositories listed
-
Natural Language Guided Visual Relationship Detection16 Nov 2017 0 repositories listed
-
28 Jul 2017 0 repositories listed
-
Care about you: towards large-scale human-centric visual relationship detection28 May 2017 0 repositories listed
-
ViP-CNN: Visual Phrase Guided Convolutional Neural Network23 Feb 2017 0 repositories listed
-
31 Jul 2016 0 repositories listed
Syntology lines on 9 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections