{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/robustness-via-retrying-closed-loop-robotic","title":"Robustness via Retrying: Closed-Loop Robotic Manipulation with Self-Supervised Learning","arxiv_id":"1810.03043","date":"2018-10-06","proceeding":null,"authors":["Frederik Ebert","Sudeep Dasari","Alex X. Lee","Sergey Levine","Chelsea Finn"],"abstract":"Prediction is an appealing objective for self-supervised learning of\nbehavioral skills, particularly for autonomous robots. However, effectively\nutilizing predictive models for control, especially with raw image inputs,\nposes a number of major challenges. How should the predictions be used? What\nhappens when they are inaccurate? In this paper, we tackle these questions by\nproposing a method for learning robotic skills from raw image observations,\nusing only autonomously collected experience. We show that even an imperfect\nmodel can complete complex tasks if it can continuously retry, but this\nrequires the model to not lose track of the objective (e.g., the object of\ninterest). To enable a robot to continuously retry a task, we devise a\nself-supervised algorithm for learning image registration, which can keep track\nof objects of interest for the duration of the trial. We demonstrate that this\nidea can be combined with a video-prediction based controller to enable complex\nbehaviors to be learned from scratch using only raw visual inputs, including\ngrasping, repositioning objects, and non-prehensile manipulation. Our\nreal-world experiments demonstrate that a model trained with 160 robot hours of\nautonomously collected, unlabeled data is able to successfully perform complex\nmanipulation tasks with a wide range of objects not seen during training.","url_abs":"http://arxiv.org/abs/1810.03043v1","url_pdf":"http://arxiv.org/pdf/1810.03043v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"robustness-via-retrying-closed-loop-robotic","repo_url":"https://github.com/febert/neuralproposal_cem_classproject","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"robustness-via-retrying-closed-loop-robotic","repo_url":"https://github.com/febert/robustness_via_retrying","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"robustness-via-retrying-closed-loop-robotic","repo_url":"https://github.com/shijiwensjw/vp_based_control","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"image-registration","task_name":"Image Registration"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"video-prediction","task_name":"Video Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1810.03043","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}