{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-to-navigate-unseen-environments-back","title":"Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout","arxiv_id":"1904.04195","date":"2019-04-08","proceeding":"NAACL 2019 6","authors":["Hao Tan","Licheng Yu","Mohit Bansal"],"abstract":"A grand goal in AI is to build a robot that can accurately navigate based on\nnatural language instructions, which requires the agent to perceive the scene,\nunderstand and ground language, and act in the real-world environment. One key\nchallenge here is to learn to navigate in new environments that are unseen\nduring training. Most of the existing approaches perform dramatically worse in\nunseen environments as compared to seen ones. In this paper, we present a\ngeneralizable navigational agent. Our agent is trained in two stages. The first\nstage is training via mixed imitation and reinforcement learning, combining the\nbenefits from both off-policy and on-policy optimization. The second stage is\nfine-tuning via newly-introduced 'unseen' triplets (environment, path,\ninstruction). To generate these unseen triplets, we propose a simple but\neffective 'environmental dropout' method to mimic unseen environments, which\novercomes the problem of limited seen environment variability. Next, we apply\nsemi-supervised learning (via back-translation) on these dropped-out\nenvironments to generate new paths and instructions. Empirically, we show that\nour agent is substantially better at generalizability when fine-tuned with\nthese triplets, outperforming the state-of-art approaches by a large margin on\nthe private unseen test set of the Room-to-Room task, and achieving the top\nrank on the leaderboard.","url_abs":"http://arxiv.org/abs/1904.04195v1","url_pdf":"http://arxiv.org/pdf/1904.04195v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-to-navigate-unseen-environments-back","repo_url":"https://github.com/airsplay/R2R-EnvDrop","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"navigate","task_name":"Navigate"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"translation","task_name":"Translation"},{"task_slug":"vision-language-navigation","task_name":"Vision-Language Navigation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/vision-and-language-navigation-on-vln","task":"Vision and Language Navigation","dataset":"VLN Challenge","model":"null","rank_in_archive_order":24,"of":145,"metrics":{"error":"3.26","length":"686.82","oracle success":"0.99","spl":"0.01","success":"0.69"},"uses_additional_data":false},{"leaderboard":"/sota/vision-and-language-navigation-on-vln","task":"Vision and Language Navigation","dataset":"VLN Challenge","model":"Back Translation with Environmental Dropout (no beam search)","rank_in_archive_order":111,"of":145,"metrics":{"error":"5.23","length":"11.66","oracle success":"0.59","spl":"0.47","success":"0.51"},"uses_additional_data":false},{"leaderboard":"/sota/vision-language-navigation-on-room2room","task":"Vision-Language Navigation","dataset":"Room2Room","model":"R2R+EnvDrop","rank_in_archive_order":1,"of":3,"metrics":{"spl":"0.61"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1904.04195","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1904.04195"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/airsplay/R2R-EnvDrop","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":4},"by_repo_kind":{"official":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"de363b1cbf20852f","entry":"build_vocab","repo":"airsplay/R2R-EnvDrop","repo_kind":"official","path":"r2r_src/utils.py","file_url":"https://github.com/airsplay/R2R-EnvDrop/blob/HEAD/r2r_src/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"de363b1cbf20852f"}},{"code_sha256_prefix":"7b5b4685db18fbca","entry":"compute_bleu","repo":"airsplay/R2R-EnvDrop","repo_kind":"official","path":"r2r_src/bleu.py","file_url":"https://github.com/airsplay/R2R-EnvDrop/blob/HEAD/r2r_src/bleu.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7b5b4685db18fbca"}},{"code_sha256_prefix":"1d9c8c3cd78345c4","entry":"load_datasets","repo":"airsplay/R2R-EnvDrop","repo_kind":"official","path":"r2r_src/utils.py","file_url":"https://github.com/airsplay/R2R-EnvDrop/blob/HEAD/r2r_src/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1d9c8c3cd78345c4"}},{"code_sha256_prefix":"e3a90df1a875a38e","entry":"load_nav_graphs","repo":"airsplay/R2R-EnvDrop","repo_kind":"official","path":"r2r_src/utils.py","file_url":"https://github.com/airsplay/R2R-EnvDrop/blob/HEAD/r2r_src/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e3a90df1a875a38e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}