{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2609-02350","title":"LookStep: Efficient Vision-Language Navigation with Linguistic Foresight and Event Driven Memory","arxiv_id":"2609.02350","date":"2026-09-02","proceeding":null,"authors":["Kun-Yang Yu","Yingzhe Li","Hongyu Xu","Shi-Yu Tian","Zhi Zhou","Yang Chen","Ming Yang","Sheng Wang","Qing Yu","Lan-Zhe Guo","Yu-Feng Li"],"abstract":"Vision-Language Navigation (VLN) requires an embodied agent to follow natural-language instructions in unseen environments. Recent progress has been largely driven by Multimodal Large Language Models (MLLMs). Existing methods follow a next-step action prediction paradigm, supervising only the expert action, which requires a high quantity of data for training. They also rely on cognitive maps, accumulated historical frames, or external 3D tools to maintain states, leading to high computational and memory overhead. To realize resource efficiency VLN, we propose LookStep, a unified end-to-end framework that combines Language Centric Future State Modeling and Event Driven Rolling Memory that uses language labels to generate coarse-grained navigation progress and future states for each candidate action, while autonomously deciding whether to write each observation into a bounded rolling memory with a semantic role. We validate LookStep empirically. On VLN-CE tasks, LookStep outperforms existing methods under the same training settings, achieving a 49.7\\% success rate on R2R-CE Val-Unseen with better memory efficiency and less data usage. Code and model is available at https://github.com/kunyang-YU/LookStep.","url_abs":"https://arxiv.org/abs/2609.02350","url_pdf":"https://arxiv.org/pdf/2609.02350","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2609.02350","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2609.02350"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/kunyang-YU/LookStep","reach":{"status":"ok"}}],"summary":{"unverified":15},"by_repo_kind":{"found_in_text":{"samples":15,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":15,"samples":[{"code_sha256_prefix":"05126c2bfca7e26e","entry":"collect_episode_ids","repo":"kunyang-YU/LookStep","repo_kind":"found_in_text","path":"model_training/prepare_ms_swift_data.py","file_url":"https://github.com/kunyang-YU/LookStep/blob/HEAD/model_training/prepare_ms_swift_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"05126c2bfca7e26e"}},{"code_sha256_prefix":"190a0b7e88b964c3","entry":"compare","repo":"kunyang-YU/LookStep","repo_kind":"found_in_text","path":"simulation/verify_paper_results.py","file_url":"https://github.com/kunyang-YU/LookStep/blob/HEAD/simulation/verify_paper_results.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"190a0b7e88b964c3"}},{"code_sha256_prefix":"d0817f75482203f6","entry":"existing_dir","repo":"kunyang-YU/LookStep","repo_kind":"found_in_text","path":"check_setup.py","file_url":"https://github.com/kunyang-YU/LookStep/blob/HEAD/check_setup.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d0817f75482203f6"}},{"code_sha256_prefix":"a6397c9d6eaa2848","entry":"existing_file","repo":"kunyang-YU/LookStep","repo_kind":"found_in_text","path":"check_setup.py","file_url":"https://github.com/kunyang-YU/LookStep/blob/HEAD/check_setup.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a6397c9d6eaa2848"}},{"code_sha256_prefix":"cb2834087368ba0c","entry":"get_instruction_text","repo":"kunyang-YU/LookStep","repo_kind":"found_in_text","path":"data_construction/build_short_label_dataset.py","file_url":"https://github.com/kunyang-YU/LookStep/blob/HEAD/data_construction/build_short_label_dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cb2834087368ba0c"}},{"code_sha256_prefix":"fe99e3a793f9f15c","entry":"load","repo":"kunyang-YU/LookStep","repo_kind":"found_in_text","path":"simulation/verify_paper_results.py","file_url":"https://github.com/kunyang-YU/LookStep/blob/HEAD/simulation/verify_paper_results.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fe99e3a793f9f15c"}},{"code_sha256_prefix":"08b57cfd801ce772","entry":"normalize_dtype","repo":"kunyang-YU/LookStep","repo_kind":"found_in_text","path":"simulation/evaluate_short_label_sim.py","file_url":"https://github.com/kunyang-YU/LookStep/blob/HEAD/simulation/evaluate_short_label_sim.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"08b57cfd801ce772"}},{"code_sha256_prefix":"e4b40dac46620f09","entry":"normalize_image_path","repo":"kunyang-YU/LookStep","repo_kind":"found_in_text","path":"model_training/prepare_ms_swift_data.py","file_url":"https://github.com/kunyang-YU/LookStep/blob/HEAD/model_training/prepare_ms_swift_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e4b40dac46620f09"}},{"code_sha256_prefix":"c271350561c8e5fd","entry":"normalize_label","repo":"kunyang-YU/LookStep","repo_kind":"found_in_text","path":"simulation/evaluate_short_label_sim.py","file_url":"https://github.com/kunyang-YU/LookStep/blob/HEAD/simulation/evaluate_short_label_sim.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c271350561c8e5fd"}},{"code_sha256_prefix":"17d76d444c468257","entry":"normalize_output_path","repo":"kunyang-YU/LookStep","repo_kind":"found_in_text","path":"data_construction/build_short_label_dataset.py","file_url":"https://github.com/kunyang-YU/LookStep/blob/HEAD/data_construction/build_short_label_dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"17d76d444c468257"}},{"code_sha256_prefix":"fdf06545d7aa50cb","entry":"normalized_metrics","repo":"kunyang-YU/LookStep","repo_kind":"found_in_text","path":"simulation/verify_paper_results.py","file_url":"https://github.com/kunyang-YU/LookStep/blob/HEAD/simulation/verify_paper_results.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fdf06545d7aa50cb"}},{"code_sha256_prefix":"90611a6b2cecec1d","entry":"python_version_check","repo":"kunyang-YU/LookStep","repo_kind":"found_in_text","path":"check_setup.py","file_url":"https://github.com/kunyang-YU/LookStep/blob/HEAD/check_setup.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"90611a6b2cecec1d"}},{"code_sha256_prefix":"8463cb2cbfc33b14","entry":"read_json_or_gzip","repo":"kunyang-YU/LookStep","repo_kind":"found_in_text","path":"data_construction/build_short_label_dataset.py","file_url":"https://github.com/kunyang-YU/LookStep/blob/HEAD/data_construction/build_short_label_dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8463cb2cbfc33b14"}},{"code_sha256_prefix":"a59d894a229a574d","entry":"resolve_image_path","repo":"kunyang-YU/LookStep","repo_kind":"found_in_text","path":"model_training/prepare_ms_swift_data.py","file_url":"https://github.com/kunyang-YU/LookStep/blob/HEAD/model_training/prepare_ms_swift_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a59d894a229a574d"}},{"code_sha256_prefix":"b0bad42cce5c9a0f","entry":"strip_thinking","repo":"kunyang-YU/LookStep","repo_kind":"found_in_text","path":"simulation/evaluate_short_label_sim.py","file_url":"https://github.com/kunyang-YU/LookStep/blob/HEAD/simulation/evaluate_short_label_sim.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b0bad42cce5c9a0f"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CV","source":"arxiv_daily_20260903.json"},"syntology_extracted_results":null}