{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2606-00095","title":"Bridging the 2D-3D Gap: A Hierarchical Semantic-Geometric Map for Vision Language Navigation","arxiv_id":"2606.00095","date":"2026-05-25","proceeding":null,"authors":["Kailing Li","Tianwen Qian","Lijin Yang","Yuqian Fu","Jingyu Gong","Xiaoling Wang","Liang He"],"abstract":"Vision-Language Navigation (VLN) enables embodied agents to reach target locations in unseen environments by following language instructions. Despite recent progress with vision-language models (VLMs), a critical semantic-geometric gap remains: while VLMs excel at language and 2D visual understanding, they struggle with 3D spatial reasoning and fail to capture the causal dynamics between actions and spatial transitions, resulting in unreliable navigation, particularly in zero-shot settings. To bridge this gap, we propose a Hierarchical Semantic-Geometric Map (HSGM) that transforms 3D geometric information into a structured representation compatible with VLMs, effectively linking them to the physical world. Specifically, HSGM is represented as a multi-channel top-down map organized into three levels: (1) geometric level that records navigable regions and obstacles, (2) semantic level that represents objects and their relations, and (3) decision level that supports high-level task reasoning and goal selection. During navigation, the VLM acts as a high-level semantic planner, interpreting the spatial layout encoded in the HSGM to select geometrically valid waypoints, while low-level, collision-free movements between waypoints are executed by a classical path-planning algorithm, fully decoupling semantic reasoning from action execution. Additionally, complex instructions are decomposed into subtasks to alleviate the problem of progress forgetting or hallucinating in long-horizon navigation. Extensive experiments on R2R-CE and RxR-CE benchmarks demonstrate that our zero-shot framework achieves state-of-the-art performance and even outperforms several supervised methods. Code is available at https://github.com/Teacher-Tom/HSGM_public.","url_abs":"https://arxiv.org/abs/2606.00095","url_pdf":"https://arxiv.org/pdf/2606.00095","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2606.00095","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2606.00095"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/Teacher-Tom/HSGM_public","reach":null}],"summary":{"ran":16},"by_repo_kind":{"found_in_text":{"samples":16,"ran":16,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"364173955bb2fccd","entry":"calculate_focal_length","repo":"Teacher-Tom/HSGM_public","repo_kind":"found_in_text","path":"src/utils.py","file_url":"https://github.com/Teacher-Tom/HSGM_public/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"364173955bb2fccd"}},{"code_sha256_prefix":"569ac508e675f80b","entry":"get_class_color","repo":"Teacher-Tom/HSGM_public","repo_kind":"found_in_text","path":"src/segmentation/instance_segmentation.py","file_url":"https://github.com/Teacher-Tom/HSGM_public/blob/HEAD/src/segmentation/instance_segmentation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"569ac508e675f80b"}},{"code_sha256_prefix":"96d90af68a987bf5","entry":"get_pointcloud_from_depth","repo":"Teacher-Tom/HSGM_public","repo_kind":"found_in_text","path":"src/mapping_utils/geometry.py","file_url":"https://github.com/Teacher-Tom/HSGM_public/blob/HEAD/src/mapping_utils/geometry.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"96d90af68a987bf5"}},{"code_sha256_prefix":"1790b9e4dc4dc878","entry":"get_pointcloud_from_depth_mask","repo":"Teacher-Tom/HSGM_public","repo_kind":"found_in_text","path":"src/mapping_utils/geometry.py","file_url":"https://github.com/Teacher-Tom/HSGM_public/blob/HEAD/src/mapping_utils/geometry.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1790b9e4dc4dc878"}},{"code_sha256_prefix":"7e3fb416779b1660","entry":"habitat_camera_intrinsic","repo":"Teacher-Tom/HSGM_public","repo_kind":"found_in_text","path":"src/mapping_utils/transform.py","file_url":"https://github.com/Teacher-Tom/HSGM_public/blob/HEAD/src/mapping_utils/transform.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7e3fb416779b1660"}},{"code_sha256_prefix":"e4dded559c35aa58","entry":"habitat_translation","repo":"Teacher-Tom/HSGM_public","repo_kind":"found_in_text","path":"src/mapping_utils/transform.py","file_url":"https://github.com/Teacher-Tom/HSGM_public/blob/HEAD/src/mapping_utils/transform.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e4dded559c35aa58"}},{"code_sha256_prefix":"3665c463b1e65415","entry":"preprocess_depth","repo":"Teacher-Tom/HSGM_public","repo_kind":"found_in_text","path":"src/mapping_utils/preprocess.py","file_url":"https://github.com/Teacher-Tom/HSGM_public/blob/HEAD/src/mapping_utils/preprocess.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3665c463b1e65415"}},{"code_sha256_prefix":"8b5e73fc5e02477a","entry":"preprocess_depth_noise","repo":"Teacher-Tom/HSGM_public","repo_kind":"found_in_text","path":"src/mapping_utils/preprocess.py","file_url":"https://github.com/Teacher-Tom/HSGM_public/blob/HEAD/src/mapping_utils/preprocess.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8b5e73fc5e02477a"}},{"code_sha256_prefix":"70b7a90cc94c219b","entry":"preprocess_image","repo":"Teacher-Tom/HSGM_public","repo_kind":"found_in_text","path":"src/mapping_utils/preprocess.py","file_url":"https://github.com/Teacher-Tom/HSGM_public/blob/HEAD/src/mapping_utils/preprocess.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"70b7a90cc94c219b"}},{"code_sha256_prefix":"9379f19a691e62df","entry":"project_frontier","repo":"Teacher-Tom/HSGM_public","repo_kind":"found_in_text","path":"src/mapping_utils/projection.py","file_url":"https://github.com/Teacher-Tom/HSGM_public/blob/HEAD/src/mapping_utils/projection.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9379f19a691e62df"}},{"code_sha256_prefix":"e8fd6df3c3573e4c","entry":"translate_grid_to_point","repo":"Teacher-Tom/HSGM_public","repo_kind":"found_in_text","path":"src/mapping_utils/projection.py","file_url":"https://github.com/Teacher-Tom/HSGM_public/blob/HEAD/src/mapping_utils/projection.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e8fd6df3c3573e4c"}},{"code_sha256_prefix":"1bfb99ecaa9a007c","entry":"translate_point_to_grid","repo":"Teacher-Tom/HSGM_public","repo_kind":"found_in_text","path":"src/mapping_utils/projection.py","file_url":"https://github.com/Teacher-Tom/HSGM_public/blob/HEAD/src/mapping_utils/projection.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1bfb99ecaa9a007c"}},{"code_sha256_prefix":"0ac967f2f3d8e8fb","entry":"translate_to_world","repo":"Teacher-Tom/HSGM_public","repo_kind":"found_in_text","path":"src/mapping_utils/geometry.py","file_url":"https://github.com/Teacher-Tom/HSGM_public/blob/HEAD/src/mapping_utils/geometry.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0ac967f2f3d8e8fb"}},{"code_sha256_prefix":"47a14f5261f20eed","entry":"visualize_instance_segmentation","repo":"Teacher-Tom/HSGM_public","repo_kind":"found_in_text","path":"src/segmentation/instance_segmentation.py","file_url":"https://github.com/Teacher-Tom/HSGM_public/blob/HEAD/src/segmentation/instance_segmentation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"47a14f5261f20eed"}},{"code_sha256_prefix":"27130b0a9e830db9","entry":"visualize_path","repo":"Teacher-Tom/HSGM_public","repo_kind":"found_in_text","path":"src/mapping_utils/path_planning.py","file_url":"https://github.com/Teacher-Tom/HSGM_public/blob/HEAD/src/mapping_utils/path_planning.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"27130b0a9e830db9"}},{"code_sha256_prefix":"4d2d80f370ee8d61","entry":"wrap_text","repo":"Teacher-Tom/HSGM_public","repo_kind":"found_in_text","path":"src/run_experiments.py","file_url":"https://github.com/Teacher-Tom/HSGM_public/blob/HEAD/src/run_experiments.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4d2d80f370ee8d61"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}