{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/grasp2vec-learning-object-representations","title":"Grasp2Vec: Learning Object Representations from Self-Supervised Grasping","arxiv_id":"1811.06964","date":"2018-11-16","proceeding":null,"authors":["Eric Jang","Coline Devin","Vincent Vanhoucke","Sergey Levine"],"abstract":"Well structured visual representations can make robot learning faster and can\nimprove generalization. In this paper, we study how we can acquire effective\nobject-centric representations for robotic manipulation tasks without human\nlabeling by using autonomous robot interaction with the environment. Such\nrepresentation learning methods can benefit from continuous refinement of the\nrepresentation as the robot collects more experience, allowing them to scale\neffectively without human intervention. Our representation learning approach is\nbased on object persistence: when a robot removes an object from a scene, the\nrepresentation of that scene should change according to the features of the\nobject that was removed. We formulate an arithmetic relationship between\nfeature vectors from this observation, and use it to learn a representation of\nscenes and objects that can then be used to identify object instances, localize\nthem in the scene, and perform goal-directed grasping tasks where the robot\nmust retrieve commanded objects from a bin. The same grasping procedure can\nalso be used to automatically collect training data for our method, by\nrecording images of scenes, grasping and removing an object, and recording the\noutcome. Our experiments demonstrate that this self-supervised approach for\ntasked grasping substantially outperforms direct reinforcement learning from\nimages and prior representation learning methods.","url_abs":"http://arxiv.org/abs/1811.06964v2","url_pdf":"http://arxiv.org/pdf/1811.06964v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"grasp2vec-learning-object-representations","repo_url":"https://github.com/google-research/tensor2robot/tree/master/research/grasp2vec","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"object","task_name":"Object"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1811.06964","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1811.06964"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-research/tensor2robot/tree/master/research/grasp2vec","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":5},"by_repo_kind":{"listed":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a2e9e959b8fc2e23","entry":"batch_norm","repo":"google-research/tensor2robot","repo_kind":"listed","path":"layers/film_resnet_model.py","file_url":"https://github.com/google-research/tensor2robot/blob/HEAD/layers/film_resnet_model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a2e9e959b8fc2e23"}},{"code_sha256_prefix":"76cc1c44e6638e42","entry":"fixed_padding","repo":"google-research/tensor2robot","repo_kind":"listed","path":"layers/film_resnet_model.py","file_url":"https://github.com/google-research/tensor2robot/blob/HEAD/layers/film_resnet_model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"76cc1c44e6638e42"}},{"code_sha256_prefix":"963af09b3fe6aa41","entry":"gaussian_mixture_approximate_mode","repo":"google-research/tensor2robot","repo_kind":"listed","path":"layers/mdn.py","file_url":"https://github.com/google-research/tensor2robot/blob/HEAD/layers/mdn.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"963af09b3fe6aa41"}},{"code_sha256_prefix":"fa932287ef756349","entry":"pfor_map_fn","repo":"google-research/tensor2robot","repo_kind":"listed","path":"meta_learning/maml_model.py","file_url":"https://github.com/google-research/tensor2robot/blob/HEAD/meta_learning/maml_model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fa932287ef756349"}},{"code_sha256_prefix":"28b4b135c21f3de2","entry":"resnet_endpoints","repo":"google-research/tensor2robot","repo_kind":"listed","path":"layers/resnet.py","file_url":"https://github.com/google-research/tensor2robot/blob/HEAD/layers/resnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"28b4b135c21f3de2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}