{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/vision-language-navigation-with-random","title":"Vision-Language Navigation with Random Environmental Mixup","arxiv_id":"2106.07876","date":"2021-06-15","proceeding":"ICCV 2021 10","authors":["Chong Liu","Fengda Zhu","Xiaojun Chang","Xiaodan Liang","ZongYuan Ge","Yi-Dong Shen"],"abstract":"Vision-language Navigation (VLN) tasks require an agent to navigate step-by-step while perceiving the visual observations and comprehending a natural language instruction. Large data bias, which is caused by the disparity ratio between the small data scale and large navigation space, makes the VLN task challenging. Previous works have proposed various data augmentation methods to reduce data bias. However, these works do not explicitly reduce the data bias across different house scenes. Therefore, the agent would overfit to the seen scenes and achieve poor navigation performance in the unseen scenes. To tackle this problem, we propose the Random Environmental Mixup (REM) method, which generates cross-connected house scenes as augmented data via mixuping environment. Specifically, we first select key viewpoints according to the room connection graph for each scene. Then, we cross-connect the key views of different scenes to construct augmented scenes. Finally, we generate augmented instruction-path pairs in the cross-connected scenes. The experimental results on benchmark datasets demonstrate that our augmentation data via REM help the agent reduce its performance gap between the seen and unseen environment and improve the overall performance, making our model the best existing approach on the standard VLN benchmark. The code have released: https://github.com/LCFractal/VLNREM.","url_abs":"https://arxiv.org/abs/2106.07876v3","url_pdf":"https://arxiv.org/pdf/2106.07876v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"vision-language-navigation-with-random","repo_url":"https://github.com/lcfractal/vlnrem","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"navigate","task_name":"Navigate"},{"task_slug":"vision-language-navigation","task_name":"Vision-Language Navigation"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dqn","method_name":"DQN"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"mixup","method_name":"Mixup"},{"method_slug":"q-learning","method_name":"Q-Learning"},{"method_slug":"rem","method_name":"REM"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/vision-and-language-navigation-on-vln","task":"Vision and Language Navigation","dataset":"VLN Challenge","model":"single-run","rank_in_archive_order":40,"of":145,"metrics":{"error":"3.87","length":"13.11","oracle success":"0.72","spl":"0.59","success":"0.65"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2106.07876","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.07876"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lcfractal/vlnrem","reach":null}],"summary":{"ran_draft_wrong":1,"unverified":3},"by_repo_kind":{"official":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"dfdde7b37c661cde","entry":"_combine_instructions","repo":"lcfractal/vlnrem","repo_kind":"official","path":"REM/REM_generate_graph.py","file_url":"https://github.com/lcfractal/vlnrem/blob/HEAD/REM/REM_generate_graph.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dfdde7b37c661cde"}},{"code_sha256_prefix":"2b6640dd55899fc8","entry":"generate_RER2R","repo":"lcfractal/vlnrem","repo_kind":"official","path":"REM/REM_generate_graph.py","file_url":"https://github.com/lcfractal/vlnrem/blob/HEAD/REM/REM_generate_graph.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2b6640dd55899fc8"}},{"code_sha256_prefix":"3cf57252760c8b3e","entry":"load","repo":"lcfractal/vlnrem","repo_kind":"official","path":"REM/REM_generate_graph.py","file_url":"https://github.com/lcfractal/vlnrem/blob/HEAD/REM/REM_generate_graph.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3cf57252760c8b3e"}},{"code_sha256_prefix":"7303d4207d2e02bb","entry":"save_single","repo":"lcfractal/vlnrem","repo_kind":"official","path":"REM/REM_generate_graph.py","file_url":"https://github.com/lcfractal/vlnrem/blob/HEAD/REM/REM_generate_graph.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7303d4207d2e02bb"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}