{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rg-san-rule-guided-spatial-awareness-network","title":"RG-SAN: Rule-Guided Spatial Awareness Network for End-to-End 3D Referring Expression Segmentation","arxiv_id":"2412.02402","date":"2024-12-03","proceeding":null,"authors":["Changli Wu","Qi Chen","Jiayi Ji","Haowei Wang","Yiwei Ma","You Huang","Gen Luo","Hao Fei","Xiaoshuai Sun","Rongrong Ji"],"abstract":"3D Referring Expression Segmentation (3D-RES) aims to segment 3D objects by correlating referring expressions with point clouds. However, traditional approaches frequently encounter issues like over-segmentation or mis-segmentation, due to insufficient emphasis on spatial information of instances. In this paper, we introduce a Rule-Guided Spatial Awareness Network (RG-SAN) by utilizing solely the spatial information of the target instance for supervision. This approach enables the network to accurately depict the spatial relationships among all entities described in the text, thus enhancing the reasoning capabilities. The RG-SAN consists of the Text-driven Localization Module (TLM) and the Rule-guided Weak Supervision (RWS) strategy. The TLM initially locates all mentioned instances and iteratively refines their positional information. The RWS strategy, acknowledging that only target objects have supervised positional information, employs dependency tree rules to precisely guide the core instance's positioning. Extensive testing on the ScanRefer benchmark has shown that RG-SAN not only establishes new performance benchmarks, with an mIoU increase of 5.1 points, but also exhibits significant improvements in robustness when processing descriptions with spatial ambiguity. All codes are available at https://github.com/sosppxo/RG-SAN.","url_abs":"https://arxiv.org/abs/2412.02402v2","url_pdf":"https://arxiv.org/pdf/2412.02402v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rg-san-rule-guided-spatial-awareness-network","repo_url":"https://github.com/sosppxo/rg-san","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"referring-expression","task_name":"Referring Expression"},{"task_slug":"referring-expression-segmentation","task_name":"Referring Expression Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2412.02402","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.02402"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/sosppxo/RG-SAN","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/ScanNet/ScanNet","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/referit3d/referit3d","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sosppxo/rg-san","reach":{"status":"ok"}}],"summary":{"ran":1,"ran_honours":1,"ran_fixture":1,"unverified":14},"by_repo_kind":{"official":{"samples":8,"ran":2,"repositories":1},"found_in_text":{"samples":9,"ran":1,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":16,"samples":[{"code_sha256_prefix":"4eb011ca450d1d87","entry":"ReferIt3DNet","repo":"referit3d/referit3d","repo_kind":"found_in_text","path":"referit3d/models/referit3d_net.py","file_url":"https://github.com/referit3d/referit3d/blob/HEAD/referit3d/models/referit3d_net.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4eb011ca450d1d87"}},{"code_sha256_prefix":"c2546a46c7f63930","entry":"scale_points","repo":"sosppxo/RG-SAN","repo_kind":"official","path":"rg_san/model/pc_util.py","file_url":"https://github.com/sosppxo/RG-SAN/blob/HEAD/rg_san/model/pc_util.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c2546a46c7f63930"}},{"code_sha256_prefix":"818099bdffc93897","entry":"shift_scale_points","repo":"sosppxo/RG-SAN","repo_kind":"official","path":"rg_san/model/pc_util.py","file_url":"https://github.com/sosppxo/RG-SAN/blob/HEAD/rg_san/model/pc_util.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"818099bdffc93897"}},{"code_sha256_prefix":"a5185a973691110d","entry":"Scene_graph_parse","repo":"sosppxo/RG-SAN","repo_kind":"official","path":"rg_san/dataset/scannetv2.py","file_url":"https://github.com/sosppxo/RG-SAN/blob/HEAD/rg_san/dataset/scannetv2.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a5185a973691110d"}},{"code_sha256_prefix":"89f8d423e0baf041","entry":"batch_dice_loss","repo":"sosppxo/RG-SAN","repo_kind":"official","path":"rg_san/model/loss.py","file_url":"https://github.com/sosppxo/RG-SAN/blob/HEAD/rg_san/model/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"89f8d423e0baf041"}},{"code_sha256_prefix":"2815d6704af5fb9c","entry":"batch_sigmoid_bce_loss","repo":"sosppxo/RG-SAN","repo_kind":"official","path":"rg_san/model/loss.py","file_url":"https://github.com/sosppxo/RG-SAN/blob/HEAD/rg_san/model/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2815d6704af5fb9c"}},{"code_sha256_prefix":"5ce41ce149ac0278","entry":"batch_sigmoid_focal_loss","repo":"sosppxo/RG-SAN","repo_kind":"official","path":"rg_san/model/loss.py","file_url":"https://github.com/sosppxo/RG-SAN/blob/HEAD/rg_san/model/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5ce41ce149ac0278"}},{"code_sha256_prefix":"3d9e947fd92c0bda","entry":"computeStatistics","repo":"ScanNet/ScanNet","repo_kind":"found_in_text","path":"Server/compute_annotation_stats.py","file_url":"https://github.com/ScanNet/ScanNet/blob/HEAD/Server/compute_annotation_stats.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"3d9e947fd92c0bda"}},{"code_sha256_prefix":"2c035a1645e66b12","entry":"getRecord","repo":"ScanNet/ScanNet","repo_kind":"found_in_text","path":"Server/compute_timings.py","file_url":"https://github.com/ScanNet/ScanNet/blob/HEAD/Server/compute_timings.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"2c035a1645e66b12"}},{"code_sha256_prefix":"ab111a546696519e","entry":"getTotal","repo":"ScanNet/ScanNet","repo_kind":"found_in_text","path":"Server/compute_timings.py","file_url":"https://github.com/ScanNet/ScanNet/blob/HEAD/Server/compute_timings.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"ab111a546696519e"}},{"code_sha256_prefix":"a53bc63ebec0ee7f","entry":"loadAllAnnotations","repo":"ScanNet/ScanNet","repo_kind":"found_in_text","path":"Server/compute_annotation_stats.py","file_url":"https://github.com/ScanNet/ScanNet/blob/HEAD/Server/compute_annotation_stats.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"a53bc63ebec0ee7f"}},{"code_sha256_prefix":"a4a324f1b49cd7c6","entry":"multi_head_attention_forward","repo":"sosppxo/RG-SAN","repo_kind":"official","path":"rg_san/model/attention.py","file_url":"https://github.com/sosppxo/RG-SAN/blob/HEAD/rg_san/model/attention.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a4a324f1b49cd7c6"}},{"code_sha256_prefix":"faa9acfe80c86398","entry":"point_indices_from_group","repo":"ScanNet/ScanNet","repo_kind":"found_in_text","path":"BenchmarkScripts/ScanNet200/utils.py","file_url":"https://github.com/ScanNet/ScanNet/blob/HEAD/BenchmarkScripts/ScanNet200/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"faa9acfe80c86398"}},{"code_sha256_prefix":"b91d93c29e979019","entry":"random_sampling","repo":"sosppxo/RG-SAN","repo_kind":"official","path":"rg_san/model/pc_util.py","file_url":"https://github.com/sosppxo/RG-SAN/blob/HEAD/rg_san/model/pc_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b91d93c29e979019"}},{"code_sha256_prefix":"94aecc61e6227dca","entry":"read_plymesh","repo":"ScanNet/ScanNet","repo_kind":"found_in_text","path":"BenchmarkScripts/ScanNet200/utils.py","file_url":"https://github.com/ScanNet/ScanNet/blob/HEAD/BenchmarkScripts/ScanNet200/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"94aecc61e6227dca"}},{"code_sha256_prefix":"3904d5900b0ebdfe","entry":"splitall","repo":"ScanNet/ScanNet","repo_kind":"found_in_text","path":"BenchmarkScripts/convert2panoptic.py","file_url":"https://github.com/ScanNet/ScanNet/blob/HEAD/BenchmarkScripts/convert2panoptic.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"3904d5900b0ebdfe"}},{"code_sha256_prefix":"b72c338d0b80682b","entry":"voxelize_pointcloud","repo":"ScanNet/ScanNet","repo_kind":"found_in_text","path":"BenchmarkScripts/ScanNet200/utils.py","file_url":"https://github.com/ScanNet/ScanNet/blob/HEAD/BenchmarkScripts/ScanNet200/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"b72c338d0b80682b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}