{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/less-is-more-toward-zero-shot-local-scene","title":"Less is More: Toward Zero-Shot Local Scene Graph Generation via Foundation Models","arxiv_id":"2310.01356","date":"2023-10-02","proceeding":null,"authors":["Shu Zhao","Huijuan Xu"],"abstract":"Humans inherently recognize objects via selective visual perception, transform specific regions from the visual field into structured symbolic knowledge, and reason their relationships among regions based on the allocation of limited attention resources in line with humans' goals. While it is intuitive for humans, contemporary perception systems falter in extracting structural information due to the intricate cognitive abilities and commonsense knowledge required. To fill this gap, we present a new task called Local Scene Graph Generation. Distinct from the conventional scene graph generation task, which encompasses generating all objects and relationships in an image, our proposed task aims to abstract pertinent structural information with partial objects and their relationships for boosting downstream tasks that demand advanced comprehension and reasoning capabilities. Correspondingly, we introduce zEro-shot Local scEne GrAph geNeraTion (ELEGANT), a framework harnessing foundation models renowned for their powerful perception and commonsense reasoning, where collaboration and information communication among foundation models yield superior outcomes and realize zero-shot local scene graph generation without requiring labeled supervision. Furthermore, we propose a novel open-ended evaluation metric, Entity-level CLIPScorE (ECLIPSE), surpassing previous closed-set evaluation metrics by transcending their limited label space, offering a broader assessment. Experiment results show that our approach markedly outperforms baselines in the open-ended evaluation setting, and it also achieves a significant performance boost of up to 24.58% over prior methods in the close-set setting, demonstrating the effectiveness and powerful reasoning ability of our proposed framework.","url_abs":"https://arxiv.org/abs/2310.01356v1","url_pdf":"https://arxiv.org/pdf/2310.01356v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"less-is-more-toward-zero-shot-local-scene","repo_url":"https://github.com/bowen-upenn/Multi-Agent-VQA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"graph-generation","task_name":"Graph Generation"},{"task_slug":"scene-graph-generation","task_name":"Scene Graph Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.01356","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.01356"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bowen-upenn/Multi-Agent-VQA","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":7},"by_repo_kind":{"listed":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"99d84bd80c9edece","entry":"calculate_iou_batch","repo":"bowen-upenn/Multi-Agent-VQA","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/bowen-upenn/Multi-Agent-VQA/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"99d84bd80c9edece"}},{"code_sha256_prefix":"1a36aeec2b00943c","entry":"draw_heat_map","repo":"bowen-upenn/Multi-Agent-VQA","repo_kind":"listed","path":"counting.py","file_url":"https://github.com/bowen-upenn/Multi-Agent-VQA/blob/HEAD/counting.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1a36aeec2b00943c"}},{"code_sha256_prefix":"b9983bc3e2610d1b","entry":"extract_image_id","repo":"bowen-upenn/Multi-Agent-VQA","repo_kind":"listed","path":"utils_func/find_matched_rest_val.py","file_url":"https://github.com/bowen-upenn/Multi-Agent-VQA/blob/HEAD/utils_func/find_matched_rest_val.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b9983bc3e2610d1b"}},{"code_sha256_prefix":"e686d5d1729b9df1","entry":"filter_boxes_pytorch","repo":"bowen-upenn/Multi-Agent-VQA","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/bowen-upenn/Multi-Agent-VQA/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e686d5d1729b9df1"}},{"code_sha256_prefix":"5639f51ad9d50828","entry":"load_answer_list","repo":"bowen-upenn/Multi-Agent-VQA","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/bowen-upenn/Multi-Agent-VQA/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5639f51ad9d50828"}},{"code_sha256_prefix":"412476f92c11ca4e","entry":"query_clip_count","repo":"bowen-upenn/Multi-Agent-VQA","repo_kind":"listed","path":"counting.py","file_url":"https://github.com/bowen-upenn/Multi-Agent-VQA/blob/HEAD/counting.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"412476f92c11ca4e"}},{"code_sha256_prefix":"7eacd2e6da39e42f","entry":"query_sam","repo":"bowen-upenn/Multi-Agent-VQA","repo_kind":"listed","path":"detections.py","file_url":"https://github.com/bowen-upenn/Multi-Agent-VQA/blob/HEAD/detections.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7eacd2e6da39e42f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}