{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/guided-feature-transformation-gft-a-neural","title":"Guided Feature Transformation (GFT): A Neural Language Grounding Module for Embodied Agents","arxiv_id":"1805.08329","date":"2018-05-22","proceeding":null,"authors":["Haonan Yu","Xiaochen Lian","Haichao Zhang","Wei Xu"],"abstract":"Recently there has been a rising interest in training agents, embodied in\nvirtual environments, to perform language-directed tasks by deep reinforcement\nlearning. In this paper, we propose a simple but effective neural language\ngrounding module for embodied agents that can be trained end to end from\nscratch taking raw pixels, unstructured linguistic commands, and sparse rewards\nas the inputs. We model the language grounding process as a language-guided\ntransformation of visual features, where latent sentence embeddings are used as\nthe transformation matrices. In several language-directed navigation tasks that\nfeature challenging partial observability and require simple reasoning, our\nmodule significantly outperforms the state of the art. We also release\nXWorld3D, an easy-to-customize 3D environment that can potentially be modified\nto evaluate a variety of embodied agents.","url_abs":"http://arxiv.org/abs/1805.08329v2","url_pdf":"http://arxiv.org/pdf/1805.08329v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"guided-feature-transformation-gft-a-neural","repo_url":"https://github.com/PaddlePaddle/XWorld","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"paddle","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"sentence-embeddings","task_name":"Sentence Embeddings"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1805.08329","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}