{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/structured-sparse-r-cnn-for-direct-scene","title":"Structured Sparse R-CNN for Direct Scene Graph Generation","arxiv_id":"2106.10815","date":"2021-06-21","proceeding":"CVPR 2022 1","authors":["Yao Teng","LiMin Wang"],"abstract":"Scene graph generation (SGG) is to detect object pairs with their relations in an image. Existing SGG approaches often use multi-stage pipelines to decompose this task into object detection, relation graph construction, and dense or dense-to-sparse relation prediction. Instead, from a perspective on SGG as a direct set prediction, this paper presents a simple, sparse, and unified framework, termed as Structured Sparse R-CNN. The key to our method is a set of learnable triplet queries and a structured triplet detector which could be jointly optimized from the training set in an end-to-end manner. Specifically, the triplet queries encode the general prior for object pairs with their relations, and provide an initial guess of scene graphs for subsequent refinement. The triplet detector presents a cascaded architecture to progressively refine the detected scene graphs with the customized dynamic heads. In addition, to relieve the training difficulty of our method, we propose a relaxed and enhanced training strategy based on knowledge distillation from a Siamese Sparse R-CNN. We perform experiments on several datasets: Visual Genome and Open Images V4/V6, and the results demonstrate that our method achieves the state-of-the-art performance. In addition, we also perform in-depth ablation studies to provide insights on our structured modeling in triplet detector design and training strategies. The code and models are made available at https://github.com/MCG-NJU/Structured-Sparse-RCNN.","url_abs":"https://arxiv.org/abs/2106.10815v2","url_pdf":"https://arxiv.org/pdf/2106.10815v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"structured-sparse-r-cnn-for-direct-scene","repo_url":"https://github.com/mcg-nju/structured-sparse-rcnn","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"structured-sparse-r-cnn-for-direct-scene","repo_url":"https://github.com/2023-MindSpore-1/ms-code-210/tree/main/cnn_direction_model","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"structured-sparse-r-cnn-for-direct-scene","repo_url":"https://github.com/2023-MindSpore-1/ms-code-6/tree/main/cnn_direction_model","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"structured-sparse-r-cnn-for-direct-scene","repo_url":"https://github.com/2024-MindSpore-1/Code7/tree/main/cnn_direction_model","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"graph-generation","task_name":"Graph Generation"},{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"relation-prediction","task_name":"Relation Prediction"},{"task_slug":"scene-graph-generation","task_name":"Scene Graph Generation"},{"task_slug":null,"task_name":"Triplet"},{"task_slug":"graph-construction","task_name":"graph construction"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"},{"method_slug":"sparse-r-cnn","method_name":"Sparse R-CNN"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2106.10815","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.10815"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/2023-MindSpore-1/ms-code-210/tree/main/cnn_direction_model","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mcg-nju/structured-sparse-rcnn","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/2024-MindSpore-1/Code7/tree/main/cnn_direction_model","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/2023-MindSpore-1/ms-code-6/tree/main/cnn_direction_model","reach":null}],"summary":{"ran_draft_wrong":1},"by_repo_kind":{},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"fbc88f9f38569eea","entry":"build_roi_relation_head","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"fbc88f9f38569eea"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}