{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-to-poke-by-poking-experiential","title":"Learning to Poke by Poking: Experiential Learning of Intuitive Physics","arxiv_id":"1606.07419","date":"2016-06-23","proceeding":"NeurIPS 2016 12","authors":["Pulkit Agrawal","Ashvin Nair","Pieter Abbeel","Jitendra Malik","Sergey Levine"],"abstract":"We investigate an experiential learning paradigm for acquiring an internal\nmodel of intuitive physics. Our model is evaluated on a real-world robotic\nmanipulation task that requires displacing objects to target locations by\npoking. The robot gathered over 400 hours of experience by executing more than\n100K pokes on different objects. We propose a novel approach based on deep\nneural networks for modeling the dynamics of robot's interactions directly from\nimages, by jointly estimating forward and inverse models of dynamics. The\ninverse model objective provides supervision to construct informative visual\nfeatures, which the forward model can then predict and in turn regularize the\nfeature space for the inverse model. The interplay between these two objectives\ncreates useful, accurate models that can then be used for multi-step decision\nmaking. This formulation has the additional benefit that it is possible to\nlearn forward models in an abstract feature space and thus alleviate the need\nof predicting pixels. Our experiments show that this joint modeling approach\noutperforms alternative methods.","url_abs":"http://arxiv.org/abs/1606.07419v2","url_pdf":"http://arxiv.org/pdf/1606.07419v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-to-poke-by-poking-experiential","repo_url":"https://github.com/mbhenaff/EEN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1606.07419","atlas_url":"https://app.syntology.ai/?focus=1606.07419","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}