{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/location-free-scene-graph-generation","title":"Location-Free Scene Graph Generation","arxiv_id":"2303.10944","date":"2023-03-20","proceeding":null,"authors":["Ege Özsoy","Felix Holm","Mahdi Saleh","Tobias Czempiel","Chantal Pellegrini","Nassir Navab","Benjamin Busam"],"abstract":"Scene Graph Generation (SGG) is a visual understanding task, aiming to describe a scene as a graph of entities and their relationships with each other. Existing works rely on location labels in form of bounding boxes or segmentation masks, increasing annotation costs and limiting dataset expansion. Recognizing that many applications do not require location data, we break this dependency and introduce location-free scene graph generation (LF-SGG). This new task aims at predicting instances of entities, as well as their relationships, without the explicit calculation of their spatial localization. To objectively evaluate the task, the predicted and ground truth scene graphs need to be compared. We solve this NP-hard problem through an efficient branching algorithm. Additionally, we design the first LF-SGG method, Pix2SG, using autoregressive sequence modeling. We demonstrate the effectiveness of our method on three scene graph generation datasets as well as two downstream tasks, image retrieval and visual question answering, and show that our approach is competitive to existing methods while not relying on location cues.","url_abs":"https://arxiv.org/abs/2303.10944v3","url_pdf":"https://arxiv.org/pdf/2303.10944v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"location-free-scene-graph-generation","repo_url":"https://github.com/egeozsoy/LF-SGG","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"graph-generation","task_name":"Graph Generation"},{"task_slug":"image-retrieval","task_name":"Image Retrieval"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"scene-graph-generation","task_name":"Scene Graph Generation"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/scene-graph-generation-on-4d-or","task":"Scene Graph Generation","dataset":"4D-OR","model":"Pix2SG","rank_in_archive_order":3,"of":5,"metrics":{"F1":"0.90"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}