{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/end-to-end-goal-driven-web-navigation","title":"End-to-End Goal-Driven Web Navigation","arxiv_id":"1602.02261","date":"2016-02-06","proceeding":"NeurIPS 2016 12","authors":["Rodrigo Nogueira","Kyunghyun Cho"],"abstract":"We propose a goal-driven web navigation as a benchmark task for evaluating an\nagent with abilities to understand natural language and plan on partially\nobserved environments. In this challenging task, an agent navigates through a\nwebsite, which is represented as a graph consisting of web pages as nodes and\nhyperlinks as directed edges, to find a web page in which a query appears. The\nagent is required to have sophisticated high-level reasoning based on natural\nlanguages and efficient sequential decision-making capability to succeed. We\nrelease a software tool, called WebNav, that automatically transforms a website\ninto this goal-driven web navigation task, and as an example, we make WikiNav,\na dataset constructed from the English Wikipedia. We extensively evaluate\ndifferent variants of neural net based artificial agents on WikiNav and observe\nthat the proposed goal-driven web navigation well reflects the advances in\nmodels, making it a suitable benchmark for evaluating future progress.\nFurthermore, we extend the WikiNav with question-answer pairs from Jeopardy!\nand test the proposed agent based on recurrent neural networks against strong\ninverted index based search engines. The artificial agents trained on WikiNav\noutperforms the engined based approaches, demonstrating the capability of the\nproposed goal-driven navigation as a good proxy for measuring the progress in\nreal-world tasks such as focused crawling and question-answering.","url_abs":"http://arxiv.org/abs/1602.02261v2","url_pdf":"http://arxiv.org/pdf/1602.02261v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"end-to-end-goal-driven-web-navigation","repo_url":"https://github.com/nyu-dl/WebNav","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"sequential-decision-making","task_name":"Sequential Decision Making"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1602.02261","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}