{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-6-dof-grasping-interaction-via-deep","title":"Learning 6-DOF Grasping Interaction via Deep Geometry-aware 3D Representations","arxiv_id":"1708.07303","date":"2017-08-24","proceeding":null,"authors":["Xinchen Yan","Jasmine Hsu","Mohi Khansari","Yunfei Bai","Arkanath Pathak","Abhinav Gupta","James Davidson","Honglak Lee"],"abstract":"This paper focuses on the problem of learning 6-DOF grasping with a parallel\njaw gripper in simulation. We propose the notion of a geometry-aware\nrepresentation in grasping based on the assumption that knowledge of 3D\ngeometry is at the heart of interaction. Our key idea is constraining and\nregularizing grasping interaction learning through 3D geometry prediction.\nSpecifically, we formulate the learning of deep geometry-aware grasping model\nin two steps: First, we learn to build mental geometry-aware representation by\nreconstructing the scene (i.e., 3D occupancy grid) from RGBD input via\ngenerative 3D shape modeling. Second, we learn to predict grasping outcome with\nits internal geometry-aware representation. The learned outcome prediction\nmodel is used to sequentially propose grasping solutions via\nanalysis-by-synthesis optimization. Our contributions are fourfold: (1) To best\nof our knowledge, we are presenting for the first time a method to learn a\n6-DOF grasping net from RGBD input; (2) We build a grasping dataset from\ndemonstrations in virtual reality with rich sensory and interaction\nannotations. This dataset includes 101 everyday objects spread across 7\ncategories, additionally, we propose a data augmentation strategy for effective\nlearning; (3) We demonstrate that the learned geometry-aware representation\nleads to about 10 percent relative performance improvement over the baseline\nCNN on grasping objects from our dataset. (4) We further demonstrate that the\nmodel generalizes to novel viewpoints and object instances.","url_abs":"http://arxiv.org/abs/1708.07303v4","url_pdf":"http://arxiv.org/pdf/1708.07303v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-6-dof-grasping-interaction-via-deep","repo_url":"https://github.com/koryako/selfcarRaspberryPi","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"3d-geometry-prediction","task_name":"3D Geometry Prediction"},{"task_slug":"3d-shape-modeling","task_name":"3D Shape Modeling"},{"task_slug":"3d-geometry","task_name":"3D geometry"},{"task_slug":"data-augmentation","task_name":"Data Augmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}