{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/adaptive-visual-scene-understanding","title":"Adaptive Visual Scene Understanding: Incremental Scene Graph Generation","arxiv_id":"2310.01636","date":"2023-10-02","proceeding":null,"authors":["Naitik Khandelwal","Xiao Liu","Mengmi Zhang"],"abstract":"Scene graph generation (SGG) analyzes images to extract meaningful information about objects and their relationships. In the dynamic visual world, it is crucial for AI systems to continuously detect new objects and establish their relationships with existing ones. Recently, numerous studies have focused on continual learning within the domains of object detection and image recognition. However, a limited amount of research focuses on a more challenging continual learning problem in SGG. This increased difficulty arises from the intricate interactions and dynamic relationships among objects, and their associated contexts. Thus, in continual learning, SGG models are often required to expand, modify, retain, and reason scene graphs within the process of adaptive visual scene understanding. To systematically explore Continual Scene Graph Generation (CSEGG), we present a comprehensive benchmark comprising three learning regimes: relationship incremental, scene incremental, and relationship generalization. Moreover, we introduce a ``Replays via Analysis by Synthesis\" method named RAS. This approach leverages the scene graphs, decomposes and re-composes them to represent different scenes, and replays the synthesized scenes based on these compositional scene graphs. The replayed synthesized scenes act as a means to practice and refine proficiency in SGG in known and unknown environments. Our experimental results not only highlight the challenges of directly combining existing continual learning methods with SGG backbones but also demonstrate the effectiveness of our proposed approach, enhancing CSEGG efficiency while simultaneously preserving privacy and memory usage. All data and source code are publicly available online.","url_abs":"https://arxiv.org/abs/2310.01636v4","url_pdf":"https://arxiv.org/pdf/2310.01636v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"adaptive-visual-scene-understanding","repo_url":"https://github.com/zhanglab-deepneurocoglab/csegg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"continual-learning","task_name":"Continual Learning"},{"task_slug":"graph-generation","task_name":"Graph Generation"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"scene-graph-generation","task_name":"Scene Graph Generation"},{"task_slug":"scene-understanding","task_name":"Scene Understanding"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.01636","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.01636"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhanglab-deepneurocoglab/csegg","reach":null}],"summary":{"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":3,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5845fb730c570b59","entry":"generate_attributes_target","repo":"zhanglab-deepneurocoglab/csegg","repo_kind":"official","path":"cvpods/evaluation/sgg_vg_evaluation.py","file_url":"https://github.com/zhanglab-deepneurocoglab/csegg/blob/HEAD/cvpods/evaluation/sgg_vg_evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5845fb730c570b59"}},{"code_sha256_prefix":"6e8e688aff9aa09a","entry":"classic_vg_sgg_evaluation","repo":"zhanglab-deepneurocoglab/csegg","repo_kind":"official","path":"cvpods/evaluation/sgg_vg_evaluation.py","file_url":"https://github.com/zhanglab-deepneurocoglab/csegg/blob/HEAD/cvpods/evaluation/sgg_vg_evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6e8e688aff9aa09a"}},{"code_sha256_prefix":"a67362dc860e46b6","entry":"convert_relation_matrix_to_triplets","repo":"zhanglab-deepneurocoglab/csegg","repo_kind":"official","path":"cvpods/evaluation/sgg_vg_evaluation.py","file_url":"https://github.com/zhanglab-deepneurocoglab/csegg/blob/HEAD/cvpods/evaluation/sgg_vg_evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a67362dc860e46b6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}