{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/grounded-language-learning-in-a-simulated-3d","title":"Grounded Language Learning in a Simulated 3D World","arxiv_id":"1706.06551","date":"2017-06-20","proceeding":null,"authors":["Karl Moritz Hermann","Felix Hill","Simon Green","Fumin Wang","Ryan Faulkner","Hubert Soyer","David Szepesvari","Wojciech Marian Czarnecki","Max Jaderberg","Denis Teplyashin","Marcus Wainwright","Chris Apps","Demis Hassabis","Phil Blunsom"],"abstract":"We are increasingly surrounded by artificially intelligent technology that\ntakes decisions and executes actions on our behalf. This creates a pressing\nneed for general means to communicate with, instruct and guide artificial\nagents, with human language the most compelling means for such communication.\nTo achieve this in a scalable fashion, agents must be able to relate language\nto the world and to actions; that is, their understanding of language must be\ngrounded and embodied. However, learning grounded language is a notoriously\nchallenging problem in artificial intelligence research. Here we present an\nagent that learns to interpret language in a simulated 3D environment where it\nis rewarded for the successful execution of written instructions. Trained via a\ncombination of reinforcement and unsupervised learning, and beginning with\nminimal prior knowledge, the agent learns to relate linguistic symbols to\nemergent perceptual representations of its physical surroundings and to\npertinent sequences of actions. The agent's comprehension of language extends\nbeyond its prior experience, enabling it to apply familiar language to\nunfamiliar situations and to interpret entirely novel instructions. Moreover,\nthe speed with which this agent learns new words increases as its semantic\nknowledge grows. This facility for generalising and bootstrapping semantic\nknowledge indicates the potential of the present approach for reconciling\nambiguous natural language with the complexity of the physical world.","url_abs":"http://arxiv.org/abs/1706.06551v2","url_pdf":"http://arxiv.org/pdf/1706.06551v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"grounded-language-learning-in-a-simulated-3d","repo_url":"https://github.com/SophiaAr/OpenAI-final-project","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"grounded-language-learning","task_name":"Grounded language learning"}],"methods":[{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1706.06551","atlas_url":"https://app.syntology.ai/?focus=1706.06551","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}