{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/from-node-to-graph-joint-reasoning-on-visual","title":"From Node to Graph: Joint Reasoning on Visual-Semantic Relational Graph for Zero-Shot Detection","arxiv_id":null,"date":"2022-02-15","proceeding":"Winter Conference on Applications of Computer Vision (WACV) 2022 2","authors":["Hui Nie","Ruiping Wang","Xilin Chen"],"abstract":"Zero-Shot Detection (ZSD), which aims at localizing andrecognizing unseen objects in a complicated scene, usuallyleverages  the  visual  and  semantic  information  of  individ-ual  objects  alone.   However,  scene  understanding  of  hu-man exceeds recognizing individual objects separately: thecontextual information among multiple objects such as vi-sual  relational information  (e.g.   visually  similar  objects)and semantic relational information (e.g.  co-occurrences)is  helpful  for  understanding  of  visual  scene.   In  this  pa-per, we verify that contextual information plays a more im-portant  role  in  ZSD  than  in  traditional  object  detection.To  make  full  use  of  such  information,  we  propose  a  newend-to-end ZSD methodGRaphAligningNetwork (GRAN)based on graph modeling and reasoning which simultane-ously considers visual and semantic information of multipleobjects instead of individual objects.  Specifically, we for-mulate a Visual Relational Graph (VRG) and a SemanticRelational Graph (SRG), where the nodes are the objectsin the image and the semantic representations of classes re-spectively and the edges are the relevance between nodesin each graph.  To characterize mutual effect between twomodalities, the two graphs are further merged into a hetero-geneous Visual-Semantic Relational Graph (VSRG), wheremodal translators are designed for the two subgraphs to en-able modal information to transform into a common spacefor communication,  and message passing among nodes isenforced to refine their representations. Comprehensive ex-periments on MSCOCO dataset demonstrate the advantageof our method over state-of-the-arts, and qualitative anal-ysis suggests the validity of using contextual information.","url_abs":"https://ieeexplore.ieee.org/document/9706663","url_pdf":"https://openaccess.thecvf.com/content/WACV2022/papers/Nie_From_Node_To_Graph_Joint_Reasoning_on_Visual-Semantic_Relational_Graph_WACV_2022_paper.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"from-node-to-graph-joint-reasoning-on-visual","repo_url":"https://github.com/witnessai/GRAN","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"generalized-zero-shot-object-detection","task_name":"Generalized Zero-Shot Object Detection"},{"task_slug":"scene-understanding","task_name":"Scene Understanding"},{"task_slug":"zero-shot-object-detection","task_name":"Zero-Shot Object Detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/zero-shot-object-detection-on-ms-coco","task":"Zero-Shot Object Detection","dataset":"MS-COCO","model":"GRAN","rank_in_archive_order":7,"of":9,"metrics":{"Recall":"62.70","mAP":"14.90"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}