{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/zero-shot-visual-imitation","title":"Zero-Shot Visual Imitation","arxiv_id":"1804.08606","date":"2018-04-23","proceeding":"ICLR 2018 1","authors":["Deepak Pathak","Parsa Mahmoudieh","Guanghao Luo","Pulkit Agrawal","Dian Chen","Yide Shentu","Evan Shelhamer","Jitendra Malik","Alexei A. Efros","Trevor Darrell"],"abstract":"The current dominant paradigm for imitation learning relies on strong\nsupervision of expert actions to learn both 'what' and 'how' to imitate. We\npursue an alternative paradigm wherein an agent first explores the world\nwithout any expert supervision and then distills its experience into a\ngoal-conditioned skill policy with a novel forward consistency loss. In our\nframework, the role of the expert is only to communicate the goals (i.e., what\nto imitate) during inference. The learned policy is then employed to mimic the\nexpert (i.e., how to imitate) after seeing just a sequence of images\ndemonstrating the desired task. Our method is 'zero-shot' in the sense that the\nagent never has access to expert actions during training or for the task\ndemonstration at inference. We evaluate our zero-shot imitator in two\nreal-world settings: complex rope manipulation with a Baxter robot and\nnavigation in previously unseen office environments with a TurtleBot. Through\nfurther experiments in VizDoom simulation, we provide evidence that better\nmechanisms for exploration lead to learning a more capable policy which in turn\nimproves end task performance. Videos, models, and more details are available\nat https://pathak22.github.io/zeroshot-imitation/","url_abs":"http://arxiv.org/abs/1804.08606v1","url_pdf":"http://arxiv.org/pdf/1804.08606v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"zero-shot-visual-imitation","repo_url":"https://github.com/pathak22/zeroshot-imitation","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"BSD-2-Clause"}}],"tasks":[{"task_slug":"imitation-learning","task_name":"Imitation Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1804.08606","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}