{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dense-object-nets-learning-dense-visual","title":"Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation","arxiv_id":"1806.08756","date":"2018-06-22","proceeding":null,"authors":["Peter R. Florence","Lucas Manuelli","Russ Tedrake"],"abstract":"What is the right object representation for manipulation? We would like\nrobots to visually perceive scenes and learn an understanding of the objects in\nthem that (i) is task-agnostic and can be used as a building block for a\nvariety of manipulation tasks, (ii) is generally applicable to both rigid and\nnon-rigid objects, (iii) takes advantage of the strong priors provided by 3D\nvision, and (iv) is entirely learned from self-supervision. This is hard to\nachieve with previous methods: much recent work in grasping does not extend to\ngrasping specific objects or other tasks, whereas task-specific learning may\nrequire many trials to generalize well across object configurations or other\ntasks. In this paper we present Dense Object Nets, which build on recent\ndevelopments in self-supervised dense descriptor learning, as a consistent\nobject representation for visual understanding and manipulation. We demonstrate\nthey can be trained quickly (approximately 20 minutes) for a wide variety of\npreviously unseen and potentially non-rigid objects. We additionally present\nnovel contributions to enable multi-object descriptor learning, and show that\nby modifying our training procedure, we can either acquire descriptors which\ngeneralize across classes of objects, or descriptors that are distinct for each\nobject instance. Finally, we demonstrate the novel application of learned dense\ndescriptors to robotic manipulation. We demonstrate grasping of specific points\non an object across potentially deformed object configurations, and demonstrate\nusing class general descriptors to transfer specific grasps across objects in a\nclass.","url_abs":"http://arxiv.org/abs/1806.08756v2","url_pdf":"http://arxiv.org/pdf/1806.08756v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dense-object-nets-learning-dense-visual","repo_url":"https://github.com/RobotLocomotion/pytorch-dense-correspondence","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"dense-object-nets-learning-dense-visual","repo_url":"https://github.com/purplearrow/pytorch-dense-correspondence","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"dense-object-nets-learning-dense-visual","repo_url":"https://github.com/venkatsai249/robo_locomotion","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"object","task_name":"Object"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1806.08756","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}