{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/webvln-vision-and-language-navigation-on","title":"WebVLN: Vision-and-Language Navigation on Websites","arxiv_id":"2312.15820","date":"2023-12-25","proceeding":null,"authors":["Qi Chen","Dileepa Pitawela","Chongyang Zhao","Gengze Zhou","Hsiang-Ting Chen","Qi Wu"],"abstract":"Vision-and-Language Navigation (VLN) task aims to enable AI agents to accurately understand and follow natural language instructions to navigate through real-world environments, ultimately reaching specific target locations. We recognise a promising opportunity to extend VLN to a comparable navigation task that holds substantial significance in our daily lives, albeit within the virtual realm: navigating websites on the Internet. This paper proposes a new task named Vision-and-Language Navigation on Websites (WebVLN), where we use question-based instructions to train an agent, emulating how users naturally browse websites. Unlike the existing VLN task that only pays attention to vision and instruction (language), the WebVLN agent further considers underlying web-specific content like HTML, which could not be seen on the rendered web pages yet contains rich visual and textual information. Toward this goal, we contribute a dataset, WebVLN-v1, and introduce a novel approach called Website-aware VLN Network (WebVLN-Net), which is built upon the foundation of state-of-the-art VLN techniques. Experimental results show that WebVLN-Net outperforms current VLN and web-related navigation methods. We believe that the introduction of the new WebVLN task and its dataset will establish a new dimension within the VLN domain and contribute to the broader vision-and-language research community. The code is available at: https://github.com/WebVLN/WebVLN.","url_abs":"https://arxiv.org/abs/2312.15820v1","url_pdf":"https://arxiv.org/pdf/2312.15820v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"webvln-vision-and-language-navigation-on","repo_url":"https://github.com/webvln/webvln","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"navigate","task_name":"Navigate"},{"task_slug":"vision-and-language-navigation","task_name":"Vision and Language Navigation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2312.15820","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.15820"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/webvln/webvln","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":1,"ran":6,"ran_honours":1},"by_repo_kind":{"official":{"samples":8,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"0f786c407fb1ee4c","entry":"swish","repo":"webvln/webvln","repo_kind":"official","path":"r2r_src/vlnbert/vlnbert_PREVALENT.py","file_url":"https://github.com/webvln/webvln/blob/HEAD/r2r_src/vlnbert/vlnbert_PREVALENT.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0f786c407fb1ee4c"}},{"code_sha256_prefix":"2b33d0ff2a8e6924","entry":"file2list","repo":"webvln/webvln","repo_kind":"official","path":"r2r_src/calculate_wups.py","file_url":"https://github.com/webvln/webvln/blob/HEAD/r2r_src/calculate_wups.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2b33d0ff2a8e6924"}},{"code_sha256_prefix":"a5da7422371d6131","entry":"fuzzy_set_membership_measure","repo":"webvln/webvln","repo_kind":"official","path":"r2r_src/calculate_wups.py","file_url":"https://github.com/webvln/webvln/blob/HEAD/r2r_src/calculate_wups.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a5da7422371d6131"}},{"code_sha256_prefix":"fdc64f4c72036ae4","entry":"gelu","repo":"webvln/webvln","repo_kind":"official","path":"r2r_src/vlnbert/vlnbert_PREVALENT.py","file_url":"https://github.com/webvln/webvln/blob/HEAD/r2r_src/vlnbert/vlnbert_PREVALENT.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fdc64f4c72036ae4"}},{"code_sha256_prefix":"990bc425ba2d9d20","entry":"items2list","repo":"webvln/webvln","repo_kind":"official","path":"r2r_src/calculate_wups.py","file_url":"https://github.com/webvln/webvln/blob/HEAD/r2r_src/calculate_wups.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"990bc425ba2d9d20"}},{"code_sha256_prefix":"b66b7866c664dff2","entry":"load_datasets","repo":"webvln/webvln","repo_kind":"official","path":"r2r_src/utils.py","file_url":"https://github.com/webvln/webvln/blob/HEAD/r2r_src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b66b7866c664dff2"}},{"code_sha256_prefix":"48d5c599339233fa","entry":"pad_answer_tokens","repo":"webvln/webvln","repo_kind":"official","path":"r2r_src/utils.py","file_url":"https://github.com/webvln/webvln/blob/HEAD/r2r_src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"48d5c599339233fa"}},{"code_sha256_prefix":"6c419fc4d3e78d25","entry":"pad_instr_tokens","repo":"webvln/webvln","repo_kind":"official","path":"r2r_src/utils.py","file_url":"https://github.com/webvln/webvln/blob/HEAD/r2r_src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6c419fc4d3e78d25"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}