{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-to-adapt-in-dynamic-real-world","title":"Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning","arxiv_id":"1803.11347","date":"2018-03-30","proceeding":"ICLR 2019 5","authors":["Anusha Nagabandi","Ignasi Clavera","Simin Liu","Ronald S. Fearing","Pieter Abbeel","Sergey Levine","Chelsea Finn"],"abstract":"Although reinforcement learning methods can achieve impressive results in\nsimulation, the real world presents two major challenges: generating samples is\nexceedingly expensive, and unexpected perturbations or unseen situations cause\nproficient but specialized policies to fail at test time. Given that it is\nimpractical to train separate policies to accommodate all situations the agent\nmay see in the real world, this work proposes to learn how to quickly and\neffectively adapt online to new tasks. To enable sample-efficient learning, we\nconsider learning online adaptation in the context of model-based reinforcement\nlearning. Our approach uses meta-learning to train a dynamics model prior such\nthat, when combined with recent data, this prior can be rapidly adapted to the\nlocal context. Our experiments demonstrate online adaptation for continuous\ncontrol tasks on both simulated and real-world agents. We first show simulated\nagents adapting their behavior online to novel terrains, crippled body parts,\nand highly-dynamic environments. We also illustrate the importance of\nincorporating online adaptation into autonomous agents that operate in the real\nworld by applying our method to a real dynamic legged millirobot. We\ndemonstrate the agent's learned ability to quickly adapt online to a missing\nleg, adjust to novel terrains and slopes, account for miscalibration or errors\nin pose estimation, and compensate for pulling payloads.","url_abs":"http://arxiv.org/abs/1803.11347v6","url_pdf":"http://arxiv.org/pdf/1803.11347v6.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-to-adapt-in-dynamic-real-world","repo_url":"https://github.com/iclavera/learning_to_adapt","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"learning-to-adapt-in-dynamic-real-world","repo_url":"https://github.com/jesbu1/carl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"continuous-control","task_name":"Continuous Control"},{"task_slug":"meta-reinforcement-learning","task_name":"Meta Reinforcement Learning"},{"task_slug":"meta-learning","task_name":"Meta-Learning"},{"task_slug":"model-based-reinforcement-learning","task_name":"Model-based Reinforcement Learning"},{"task_slug":"pose-estimation","task_name":"Pose Estimation"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"continuous-control","task_name":"continuous-control"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1803.11347","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}