{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/meta-learning-shared-hierarchies","title":"Meta Learning Shared Hierarchies","arxiv_id":"1710.09767","date":"2017-10-26","proceeding":"ICLR 2018 1","authors":["Kevin Frans","Jonathan Ho","Xi Chen","Pieter Abbeel","John Schulman"],"abstract":"We develop a metalearning approach for learning hierarchically structured\npolicies, improving sample efficiency on unseen tasks through the use of shared\nprimitives---policies that are executed for large numbers of timesteps.\nSpecifically, a set of primitives are shared within a distribution of tasks,\nand are switched between by task-specific policies. We provide a concrete\nmetric for measuring the strength of such hierarchies, leading to an\noptimization problem for quickly reaching high reward on unseen tasks. We then\npresent an algorithm to solve this problem end-to-end through the use of any\noff-the-shelf reinforcement learning method, by repeatedly sampling new tasks\nand resetting task-specific policies. We successfully discover meaningful motor\nprimitives for the directional movement of four-legged robots, solely by\ninteracting with distributions of mazes. We also demonstrate the\ntransferability of primitives to solve long-timescale sparse-reward obstacle\ncourses, and we enable 3D humanoid robots to robustly walk and crawl with the\nsame policy.","url_abs":"http://arxiv.org/abs/1710.09767v1","url_pdf":"http://arxiv.org/pdf/1710.09767v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"meta-learning-shared-hierarchies","repo_url":"https://github.com/openai/mlsh","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"meta-learning-shared-hierarchies","repo_url":"https://github.com/dsapandora/s_cera","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"GPL-3.0"}},{"paper_slug":"meta-learning-shared-hierarchies","repo_url":"https://github.com/roop-pal/Meta-Learning-for-StarCraft-II-Minigames","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}}],"tasks":[{"task_slug":"meta-learning","task_name":"Meta-Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1710.09767","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}