{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/listen-attend-and-walk-neural-mapping-of","title":"Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences","arxiv_id":"1506.04089","date":"2015-06-12","proceeding":null,"authors":["Hongyuan Mei","Mohit Bansal","Matthew R. Walter"],"abstract":"We propose a neural sequence-to-sequence model for direction following, a\ntask that is essential to realizing effective autonomous agents. Our\nalignment-based encoder-decoder model with long short-term memory recurrent\nneural networks (LSTM-RNN) translates natural language instructions to action\nsequences based upon a representation of the observable world state. We\nintroduce a multi-level aligner that empowers our model to focus on sentence\n\"regions\" salient to the current world state by using multiple abstractions of\nthe input sentence. In contrast to existing methods, our model uses no\nspecialized linguistic resources (e.g., parsers) or task-specific annotations\n(e.g., seed lexicons). It is therefore generalizable, yet still achieves the\nbest results reported to-date on a benchmark single-sentence dataset and\ncompetitive results for the limited-training multi-sentence setting. We analyze\nour model through a series of ablations that elucidate the contributions of the\nprimary components of our model.","url_abs":"http://arxiv.org/abs/1506.04089v4","url_pdf":"http://arxiv.org/pdf/1506.04089v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"listen-attend-and-walk-neural-mapping-of","repo_url":"https://github.com/HMEIatJHU/NeuralWalker","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"natural-language-understanding","task_name":"Natural Language Understanding"},{"task_slug":"sentence","task_name":"Sentence"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1506.04089","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}