{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/lana-a-language-capable-navigator-for","title":"Lana: A Language-Capable Navigator for Instruction Following and Generation","arxiv_id":"2303.08409","date":"2023-03-15","proceeding":"CVPR 2023 1","authors":["Xiaohan Wang","Wenguan Wang","Jiayi Shao","Yi Yang"],"abstract":"Recently, visual-language navigation (VLN) -- entailing robot agents to follow navigation instructions -- has shown great advance. However, existing literature put most emphasis on interpreting instructions into actions, only delivering \"dumb\" wayfinding agents. In this article, we devise LANA, a language-capable navigation agent which is able to not only execute human-written navigation commands, but also provide route descriptions to humans. This is achieved by simultaneously learning instruction following and generation with only one single model. More specifically, two encoders, respectively for route and language encoding, are built and shared by two decoders, respectively, for action prediction and instruction generation, so as to exploit cross-task knowledge and capture task-specific characteristics. Throughout pretraining and fine-tuning, both instruction following and generation are set as optimization objectives. We empirically verify that, compared with recent advanced task-specific solutions, LANA attains better performances on both instruction following and route description, with nearly half complexity. In addition, endowed with language generation capability, LANA can explain to humans its behaviors and assist human's wayfinding. This work is expected to foster future efforts towards building more trustworthy and socially-intelligent navigation robots.","url_abs":"https://arxiv.org/abs/2303.08409v1","url_pdf":"https://arxiv.org/pdf/2303.08409v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"lana-a-language-capable-navigator-for","repo_url":"https://github.com/wxh1996/lana-vln","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"instruction-following","task_name":"Instruction Following"},{"task_slug":"text-generation","task_name":"Text Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2303.08409","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.08409"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wxh1996/lana-vln","reach":null}],"summary":{"ran":11,"unverified":4},"by_repo_kind":{"official":{"samples":15,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"95201a35b73fcc8b","entry":"ActionEmbeddings","repo":"wxh1996/lana-vln","repo_kind":"official","path":"finetune_src/models/vilmodel_cmt_lana.py","file_url":"https://github.com/wxh1996/lana-vln/blob/HEAD/finetune_src/models/vilmodel_cmt_lana.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"95201a35b73fcc8b"}},{"code_sha256_prefix":"972bc2726cd67bc8","entry":"BertAttention","repo":"wxh1996/lana-vln","repo_kind":"official","path":"finetune_src/models/vilmodel_cmt_lana.py","file_url":"https://github.com/wxh1996/lana-vln/blob/HEAD/finetune_src/models/vilmodel_cmt_lana.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"972bc2726cd67bc8"}},{"code_sha256_prefix":"46726083e44b793d","entry":"BertLMPredictionHead","repo":"wxh1996/lana-vln","repo_kind":"official","path":"finetune_src/models/vilmodel_cmt_lana.py","file_url":"https://github.com/wxh1996/lana-vln/blob/HEAD/finetune_src/models/vilmodel_cmt_lana.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"46726083e44b793d"}},{"code_sha256_prefix":"8439e6da38dfae41","entry":"BertOnlyMLMHead","repo":"wxh1996/lana-vln","repo_kind":"official","path":"finetune_src/models/vilmodel_cmt_lana.py","file_url":"https://github.com/wxh1996/lana-vln/blob/HEAD/finetune_src/models/vilmodel_cmt_lana.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8439e6da38dfae41"}},{"code_sha256_prefix":"fbfefa916c6de2bb","entry":"BertPredictionHeadTransform","repo":"wxh1996/lana-vln","repo_kind":"official","path":"finetune_src/models/vilmodel_cmt_lana.py","file_url":"https://github.com/wxh1996/lana-vln/blob/HEAD/finetune_src/models/vilmodel_cmt_lana.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fbfefa916c6de2bb"}},{"code_sha256_prefix":"a0efcc75e5e7e5de","entry":"BertSelfAttention","repo":"wxh1996/lana-vln","repo_kind":"official","path":"finetune_src/models/vilmodel_cmt_lana.py","file_url":"https://github.com/wxh1996/lana-vln/blob/HEAD/finetune_src/models/vilmodel_cmt_lana.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a0efcc75e5e7e5de"}},{"code_sha256_prefix":"ab4507b7278676b1","entry":"InputSelfAttention","repo":"wxh1996/lana-vln","repo_kind":"official","path":"finetune_src/models/vilmodel_cmt_lana.py","file_url":"https://github.com/wxh1996/lana-vln/blob/HEAD/finetune_src/models/vilmodel_cmt_lana.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ab4507b7278676b1"}},{"code_sha256_prefix":"10473dc20e4ea297","entry":"InputXAttention","repo":"wxh1996/lana-vln","repo_kind":"official","path":"finetune_src/models/vilmodel_cmt_lana.py","file_url":"https://github.com/wxh1996/lana-vln/blob/HEAD/finetune_src/models/vilmodel_cmt_lana.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"10473dc20e4ea297"}},{"code_sha256_prefix":"e60a700608513530","entry":"ResidualAttentionBlock","repo":"wxh1996/lana-vln","repo_kind":"official","path":"finetune_src/models/vilmodel_cmt_lana.py","file_url":"https://github.com/wxh1996/lana-vln/blob/HEAD/finetune_src/models/vilmodel_cmt_lana.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e60a700608513530"}},{"code_sha256_prefix":"034d394f08c70794","entry":"Transformer","repo":"wxh1996/lana-vln","repo_kind":"official","path":"finetune_src/models/vilmodel_cmt_lana.py","file_url":"https://github.com/wxh1996/lana-vln/blob/HEAD/finetune_src/models/vilmodel_cmt_lana.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"034d394f08c70794"}},{"code_sha256_prefix":"8515b5135628725d","entry":"VisualEmbeddings","repo":"wxh1996/lana-vln","repo_kind":"official","path":"finetune_src/models/vilmodel_cmt_lana.py","file_url":"https://github.com/wxh1996/lana-vln/blob/HEAD/finetune_src/models/vilmodel_cmt_lana.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8515b5135628725d"}},{"code_sha256_prefix":"71b64ba0c2e09916","entry":"BertLayer","repo":"wxh1996/lana-vln","repo_kind":"official","path":"finetune_src/models/vilmodel_cmt_lana.py","file_url":"https://github.com/wxh1996/lana-vln/blob/HEAD/finetune_src/models/vilmodel_cmt_lana.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"71b64ba0c2e09916"}},{"code_sha256_prefix":"1aa0b58638317139","entry":"LXRTXLayer","repo":"wxh1996/lana-vln","repo_kind":"official","path":"finetune_src/models/vilmodel_cmt_lana.py","file_url":"https://github.com/wxh1996/lana-vln/blob/HEAD/finetune_src/models/vilmodel_cmt_lana.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1aa0b58638317139"}},{"code_sha256_prefix":"c7aa8c5278bebd66","entry":"LxmertEncoder","repo":"wxh1996/lana-vln","repo_kind":"official","path":"finetune_src/models/vilmodel_cmt_lana.py","file_url":"https://github.com/wxh1996/lana-vln/blob/HEAD/finetune_src/models/vilmodel_cmt_lana.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c7aa8c5278bebd66"}},{"code_sha256_prefix":"c8a9c4bd19048e94","entry":"NavCMT","repo":"wxh1996/lana-vln","repo_kind":"official","path":"finetune_src/models/vilmodel_cmt_lana.py","file_url":"https://github.com/wxh1996/lana-vln/blob/HEAD/finetune_src/models/vilmodel_cmt_lana.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c8a9c4bd19048e94"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}