{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/scaling-simulation-to-real-transfer-by","title":"Scaling simulation-to-real transfer by learning composable robot skills","arxiv_id":"1809.10253","date":"2018-09-26","proceeding":null,"authors":["Ryan Julian","Eric Heiden","Zhanpeng He","Hejia Zhang","Stefan Schaal","Joseph J. Lim","Gaurav Sukhatme","Karol Hausman"],"abstract":"We present a novel solution to the problem of simulation-to-real transfer,\nwhich builds on recent advances in robot skill decomposition. Rather than\nfocusing on minimizing the simulation-reality gap, we learn a set of diverse\npolicies that are parameterized in a way that makes them easily reusable. This\ndiversity and parameterization of low-level skills allows us to find a\ntransferable policy that is able to use combinations and variations of\ndifferent skills to solve more complex, high-level tasks. In particular, we\nfirst use simulation to jointly learn a policy for a set of low-level skills,\nand a \"skill embedding\" parameterization which can be used to compose them.\nLater, we learn high-level policies which actuate the low-level policies via\nthis skill embedding parameterization. The high-level policies encode how and\nwhen to reuse the low-level skills together to achieve specific high-level\ntasks. Importantly, our method learns to control a real robot in joint-space to\nachieve these high-level tasks with little or no on-robot time, despite the\nfact that the low-level policies may not be perfectly transferable from\nsimulation to real, and that the low-level skills were not trained on any\nexamples of high-level tasks. We illustrate the principles of our method using\ninformative simulation experiments. We then verify its usefulness for real\nrobotics problems by learning, transferring, and composing free-space and\ncontact motion skills on a Sawyer robot using only joint-space control. We\nexperiment with several techniques for composing pre-learned skills, and find\nthat our method allows us to use both learning-based approaches and efficient\nsearch-based planning to achieve high-level tasks using only pre-learned\nskills.","url_abs":"http://arxiv.org/abs/1809.10253v3","url_pdf":"http://arxiv.org/pdf/1809.10253v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"scaling-simulation-to-real-transfer-by","repo_url":"https://github.com/ryanjulian/embed2learn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}