{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fine-grained-scene-graph-generation-with-data","title":"Fine-Grained Scene Graph Generation with Data Transfer","arxiv_id":"2203.11654","date":"2022-03-22","proceeding":null,"authors":["Ao Zhang","Yuan YAO","Qianyu Chen","Wei Ji","Zhiyuan Liu","Maosong Sun","Tat-Seng Chua"],"abstract":"Scene graph generation (SGG) is designed to extract (subject, predicate, object) triplets in images. Recent works have made a steady progress on SGG, and provide useful tools for high-level vision and language understanding. However, due to the data distribution problems including long-tail distribution and semantic ambiguity, the predictions of current SGG models tend to collapse to several frequent but uninformative predicates (e.g., on, at), which limits practical application of these models in downstream tasks. To deal with the problems above, we propose a novel Internal and External Data Transfer (IETrans) method, which can be applied in a plug-and-play fashion and expanded to large SGG with 1,807 predicate classes. Our IETrans tries to relieve the data distribution problem by automatically creating an enhanced dataset that provides more sufficient and coherent annotations for all predicates. By training on the enhanced dataset, a Neural Motif model doubles the macro performance while maintaining competitive micro performance. The code and data are publicly available at https://github.com/waxnkw/IETrans-SGG.pytorch.","url_abs":"https://arxiv.org/abs/2203.11654v2","url_pdf":"https://arxiv.org/pdf/2203.11654v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fine-grained-scene-graph-generation-with-data","repo_url":"https://github.com/waxnkw/ietrans-sgg.pytorch","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"fine-grained-scene-graph-generation-with-data","repo_url":"https://github.com/rlqja1107/torch-st-sgg","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"graph-generation","task_name":"Graph Generation"},{"task_slug":"predicate-classification","task_name":"Predicate Classification"},{"task_slug":"scene-graph-classification","task_name":"Scene Graph Classification"},{"task_slug":"scene-graph-detection","task_name":"Scene Graph Detection"},{"task_slug":"scene-graph-generation","task_name":"Scene Graph Generation"},{"task_slug":"unbiased-scene-graph-generation","task_name":"Unbiased Scene Graph Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/scene-graph-generation-on-visual-genome","task":"Scene Graph Generation","dataset":"Visual Genome","model":"IETrans","rank_in_archive_order":11,"of":19,"metrics":{"Recall@100":"27.2","Recall@50":"23.5","mean Recall @100":"18.0"},"uses_additional_data":false},{"leaderboard":"/sota/unbiased-scene-graph-generation-on-visual","task":"Unbiased Scene Graph Generation","dataset":"Visual Genome","model":"IETrans (MOTIFS-ResNeXt-101-FPN backbone; PredCls mode)","rank_in_archive_order":1,"of":31,"metrics":{"F@100":"44.1","mR@20":"28.9","ng-mR@20":"36.0"},"uses_additional_data":false},{"leaderboard":"/sota/unbiased-scene-graph-generation-on-visual","task":"Unbiased Scene Graph Generation","dataset":"Visual Genome","model":"IETrans (MOTIFS-ResNeXt-101-FPN backbone; SGCls mode)","rank_in_archive_order":7,"of":31,"metrics":{"F@100":"26.0","mR@20":"17.5","ng-mR@20":"21.8"},"uses_additional_data":false},{"leaderboard":"/sota/unbiased-scene-graph-generation-on-visual","task":"Unbiased Scene Graph Generation","dataset":"Visual Genome","model":"IETrans (MOTIFS-ResNeXt-101-FPN backbone; SGDet mode)","rank_in_archive_order":14,"of":31,"metrics":{"F@100":"21.7","mR@20":"10.9","ng-mR@20":"13.4"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2203.11654","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}