{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unsupervised-predictive-memory-in-a-goal","title":"Unsupervised Predictive Memory in a Goal-Directed Agent","arxiv_id":"1803.10760","date":"2018-03-28","proceeding":null,"authors":["Greg Wayne","Chia-Chun Hung","David Amos","Mehdi Mirza","Arun Ahuja","Agnieszka Grabska-Barwinska","Jack Rae","Piotr Mirowski","Joel Z. Leibo","Adam Santoro","Mevlana Gemici","Malcolm Reynolds","Tim Harley","Josh Abramson","Shakir Mohamed","Danilo Rezende","David Saxton","Adam Cain","Chloe Hillier","David Silver","Koray Kavukcuoglu","Matt Botvinick","Demis Hassabis","Timothy Lillicrap"],"abstract":"Animals execute goal-directed behaviours despite the limited range and scope\nof their sensors. To cope, they explore environments and store memories\nmaintaining estimates of important information that is not presently available.\nRecently, progress has been made with artificial intelligence (AI) agents that\nlearn to perform tasks from sensory input, even at a human level, by merging\nreinforcement learning (RL) algorithms with deep neural networks, and the\nexcitement surrounding these results has led to the pursuit of related ideas as\nexplanations of non-human animal learning. However, we demonstrate that\ncontemporary RL algorithms struggle to solve simple tasks when enough\ninformation is concealed from the sensors of the agent, a property called\n\"partial observability\". An obvious requirement for handling partially observed\ntasks is access to extensive memory, but we show memory is not enough; it is\ncritical that the right information be stored in the right format. We develop a\nmodel, the Memory, RL, and Inference Network (MERLIN), in which memory\nformation is guided by a process of predictive modeling. MERLIN facilitates the\nsolution of tasks in 3D virtual reality environments for which partial\nobservability is severe and memories must be maintained over long durations.\nOur model demonstrates a single learning agent architecture that can solve\ncanonical behavioural tasks in psychology and neurobiology without strong\nsimplifying assumptions about the dimensionality of sensory input or the\nduration of experiences.","url_abs":"http://arxiv.org/abs/1803.10760v1","url_pdf":"http://arxiv.org/pdf/1803.10760v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unsupervised-predictive-memory-in-a-goal","repo_url":"https://github.com/yosider/merlin","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1803.10760","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}