{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/semantic-diversity-aware-prototype-based","title":"Semantic Diversity-aware Prototype-based Learning for Unbiased Scene Graph Generation","arxiv_id":"2407.15396","date":"2024-07-22","proceeding":null,"authors":["Jaehyeong Jeon","Kibum Kim","Kanghoon Yoon","Chanyoung Park"],"abstract":"The scene graph generation (SGG) task involves detecting objects within an image and predicting predicates that represent the relationships between the objects. However, in SGG benchmark datasets, each subject-object pair is annotated with a single predicate even though a single predicate may exhibit diverse semantics (i.e., semantic diversity), existing SGG models are trained to predict the one and only predicate for each pair. This in turn results in the SGG models to overlook the semantic diversity that may exist in a predicate, thus leading to biased predictions. In this paper, we propose a novel model-agnostic Semantic Diversity-aware Prototype-based Learning (DPL) framework that enables unbiased predictions based on the understanding of the semantic diversity of predicates. Specifically, DPL learns the regions in the semantic space covered by each predicate to distinguish among the various different semantics that a single predicate can represent. Extensive experiments demonstrate that our proposed model-agnostic DPL framework brings significant performance improvement on existing SGG models, and also effectively understands the semantic diversity of predicates.","url_abs":"https://arxiv.org/abs/2407.15396v2","url_pdf":"https://arxiv.org/pdf/2407.15396v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"semantic-diversity-aware-prototype-based","repo_url":"https://github.com/jeonjaehyeong/dpl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"graph-generation","task_name":"Graph Generation"},{"task_slug":"scene-graph-generation","task_name":"Scene Graph Generation"},{"task_slug":"unbiased-scene-graph-generation","task_name":"Unbiased Scene Graph Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/unbiased-scene-graph-generation-on-visual","task":"Unbiased Scene Graph Generation","dataset":"Visual Genome","model":"DPL (MOTIFS-ResNeXt-101-FPN backbone; PredCls mode)","rank_in_archive_order":2,"of":31,"metrics":{"F@100":"44.9","mR@20":"26.2","ng-mR@20":"31.3"},"uses_additional_data":false},{"leaderboard":"/sota/unbiased-scene-graph-generation-on-visual","task":"Unbiased Scene Graph Generation","dataset":"Visual Genome","model":"DPL (MOTIFS-ResNeXt-101-FPN backbone; SGCls mode)","rank_in_archive_order":11,"of":31,"metrics":{"F@100":"25.2","mR@20":"14.1","ng-mR@20":"18.5"},"uses_additional_data":false},{"leaderboard":"/sota/unbiased-scene-graph-generation-on-visual","task":"Unbiased Scene Graph Generation","dataset":"Visual Genome","model":"DPL (MOTIFS-ResNeXt-101-FPN backbone; SGDet mode)","rank_in_archive_order":20,"of":31,"metrics":{"F@100":"20.2","mR@20":"9.4","ng-mR@20":"10.0"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.15396","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.15396"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jeonjaehyeong/dpl","reach":null}],"summary":{"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"3acba312dea9c747","entry":"show_params_status","repo":"jeonjaehyeong/dpl","repo_kind":"official","path":"tools/relation_train_net.py","file_url":"https://github.com/jeonjaehyeong/dpl/blob/HEAD/tools/relation_train_net.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"3acba312dea9c747"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}