{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/leveraging-predicate-and-triplet-learning-for","title":"Leveraging Predicate and Triplet Learning for Scene Graph Generation","arxiv_id":"2406.02038","date":"2024-06-04","proceeding":"CVPR 2024 1","authors":["Jiankai Li","Yunhong Wang","Xiefan Guo","Ruijie Yang","Weixin Li"],"abstract":"Scene Graph Generation (SGG) aims to identify entities and predict the relationship triplets \\textit{\\textless subject, predicate, object\\textgreater } in visual scenes. Given the prevalence of large visual variations of subject-object pairs even in the same predicate, it can be quite challenging to model and refine predicate representations directly across such pairs, which is however a common strategy adopted by most existing SGG methods. We observe that visual variations within the identical triplet are relatively small and certain relation cues are shared in the same type of triplet, which can potentially facilitate the relation learning in SGG. Moreover, for the long-tail problem widely studied in SGG task, it is also crucial to deal with the limited types and quantity of triplets in tail predicates. Accordingly, in this paper, we propose a Dual-granularity Relation Modeling (DRM) network to leverage fine-grained triplet cues besides the coarse-grained predicate ones. DRM utilizes contexts and semantics of predicate and triplet with Dual-granularity Constraints, generating compact and balanced representations from two perspectives to facilitate relation recognition. Furthermore, a Dual-granularity Knowledge Transfer (DKT) strategy is introduced to transfer variation from head predicates/triplets to tail ones, aiming to enrich the pattern diversity of tail classes to alleviate the long-tail problem. Extensive experiments demonstrate the effectiveness of our method, which establishes new state-of-the-art performance on Visual Genome, Open Image, and GQA datasets. Our code is available at \\url{https://github.com/jkli1998/DRM}","url_abs":"https://arxiv.org/abs/2406.02038v1","url_pdf":"https://arxiv.org/pdf/2406.02038v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"leveraging-predicate-and-triplet-learning-for","repo_url":"https://github.com/jkli1998/drm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"graph-generation","task_name":"Graph Generation"},{"task_slug":null,"task_name":"Relation"},{"task_slug":"scene-graph-generation","task_name":"Scene Graph Generation"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"},{"task_slug":null,"task_name":"Triplet"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2406.02038","atlas_url":"https://app.syntology.ai/?focus=2406.02038","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.02038"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jkli1998/drm","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2d943c0c96e61f16","entry":"entropy_loss","repo":"jkli1998/drm","repo_kind":"official","path":"maskrcnn_benchmark/layers/entropy_loss.py","file_url":"https://github.com/jkli1998/drm/blob/HEAD/maskrcnn_benchmark/layers/entropy_loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2d943c0c96e61f16"}},{"code_sha256_prefix":"2902bf4410253ff7","entry":"interpolate","repo":"jkli1998/drm","repo_kind":"official","path":"maskrcnn_benchmark/layers/misc.py","file_url":"https://github.com/jkli1998/drm/blob/HEAD/maskrcnn_benchmark/layers/misc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2902bf4410253ff7"}},{"code_sha256_prefix":"382942f3ada73679","entry":"kl_div_loss","repo":"jkli1998/drm","repo_kind":"official","path":"maskrcnn_benchmark/layers/kl_div_loss.py","file_url":"https://github.com/jkli1998/drm/blob/HEAD/maskrcnn_benchmark/layers/kl_div_loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"382942f3ada73679"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}