{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rvt-2-learning-precise-manipulation-from-few","title":"RVT-2: Learning Precise Manipulation from Few Demonstrations","arxiv_id":"2406.08545","date":"2024-06-12","proceeding":null,"authors":["Ankit Goyal","Valts Blukis","Jie Xu","Yijie Guo","Yu-Wei Chao","Dieter Fox"],"abstract":"In this work, we study how to build a robotic system that can solve multiple 3D manipulation tasks given language instructions. To be useful in industrial and household domains, such a system should be capable of learning new tasks with few demonstrations and solving them precisely. Prior works, like PerAct and RVT, have studied this problem, however, they often struggle with tasks requiring high precision. We study how to make them more effective, precise, and fast. Using a combination of architectural and system-level improvements, we propose RVT-2, a multitask 3D manipulation model that is 6X faster in training and 2X faster in inference than its predecessor RVT. RVT-2 achieves a new state-of-the-art on RLBench, improving the success rate from 65% to 82%. RVT-2 is also effective in the real world, where it can learn tasks requiring high precision, like picking up and inserting plugs, with just 10 demonstrations. Visual results, code, and trained model are provided at: https://robotic-view-transformer-2.github.io/.","url_abs":"https://arxiv.org/abs/2406.08545v1","url_pdf":"https://arxiv.org/pdf/2406.08545v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rvt-2-learning-precise-manipulation-from-few","repo_url":"https://github.com/NVlabs/RVT","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"robot-manipulation","task_name":"Robot Manipulation"},{"task_slug":"robot-manipulation-generalization","task_name":"Robot Manipulation Generalization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/robot-manipulation-on-rlbench","task":"Robot Manipulation","dataset":"RLBench","model":"RVT-2","rank_in_archive_order":5,"of":18,"metrics":{"Inference Speed (fps)":"20.6","Input Image Size":"128","Succ. Rate (18 tasks, 100 demo/task)":"81.4","Training Time (V100 x 8 x day)":"0.83"},"uses_additional_data":false},{"leaderboard":"/sota/robot-manipulation-generalization-on-gembench","task":"Robot Manipulation Generalization","dataset":"GEMBench","model":"RVT-2","rank_in_archive_order":4,"of":6,"metrics":{"Average Success Rate":"44.0","Average Success Rate (L1)":"89.1±0.8","Average Success Rate (L2)":"51.0±2.3","Average Success Rate (L3)":"36.0±2.2","Average Success Rate (L4)":"0.0±0.0"},"uses_additional_data":false},{"leaderboard":"/sota/robot-manipulation-generalization-on-the","task":"Robot Manipulation Generalization","dataset":"The COLOSSEUM","model":"RVT2","rank_in_archive_order":5,"of":9,"metrics":{"Average decrease average across all perturbations":"-19.5"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.08545","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.08545"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/NVlabs/RVT","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran":12,"ran_draft_wrong":1,"ran_violates":2},"by_repo_kind":{"official":{"samples":15,"ran":15,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":15,"samples":[{"code_sha256_prefix":"552ffd16f22068d8","entry":"add_uni_noi","repo":"NVlabs/RVT","repo_kind":"official","path":"rvt/mvt/utils.py","file_url":"https://github.com/NVlabs/RVT/blob/HEAD/rvt/mvt/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"552ffd16f22068d8"}},{"code_sha256_prefix":"a01525ed76586b6b","entry":"batched_index_select","repo":"NVlabs/RVT","repo_kind":"official","path":"rvt/mvt/renderer.py","file_url":"https://github.com/NVlabs/RVT/blob/HEAD/rvt/mvt/renderer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"a01525ed76586b6b"}},{"code_sha256_prefix":"8458f9ba0361ff43","entry":"cache_fn","repo":"NVlabs/RVT","repo_kind":"official","path":"rvt/mvt/attn.py","file_url":"https://github.com/NVlabs/RVT/blob/HEAD/rvt/mvt/attn.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8458f9ba0361ff43"}},{"code_sha256_prefix":"60fff7c3c400d7ff","entry":"default","repo":"NVlabs/RVT","repo_kind":"official","path":"rvt/mvt/attn.py","file_url":"https://github.com/NVlabs/RVT/blob/HEAD/rvt/mvt/attn.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"60fff7c3c400d7ff"}},{"code_sha256_prefix":"27f29abd3e4d0c00","entry":"eval_cls","repo":"NVlabs/RVT","repo_kind":"official","path":"rvt/models/rvt_agent.py","file_url":"https://github.com/NVlabs/RVT/blob/HEAD/rvt/models/rvt_agent.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"27f29abd3e4d0c00"}},{"code_sha256_prefix":"c78b038bba8b4c1d","entry":"eval_con","repo":"NVlabs/RVT","repo_kind":"official","path":"rvt/models/rvt_agent.py","file_url":"https://github.com/NVlabs/RVT/blob/HEAD/rvt/models/rvt_agent.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"c78b038bba8b4c1d"}},{"code_sha256_prefix":"40867edc55ab08ae","entry":"eval_con_cls","repo":"NVlabs/RVT","repo_kind":"official","path":"rvt/models/rvt_agent.py","file_url":"https://github.com/NVlabs/RVT/blob/HEAD/rvt/models/rvt_agent.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"40867edc55ab08ae"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"NVlabs/RVT","repo_kind":"official","path":"rvt/mvt/attn.py","file_url":"https://github.com/NVlabs/RVT/blob/HEAD/rvt/mvt/attn.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"f6a528fb2776e15a","entry":"normalize_quaternion","repo":"NVlabs/RVT","repo_kind":"official","path":"rvt/mvt/aug_utils.py","file_url":"https://github.com/NVlabs/RVT/blob/HEAD/rvt/mvt/aug_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"f6a528fb2776e15a"}},{"code_sha256_prefix":"09dc9e73dc9e6c80","entry":"perturb_se3","repo":"NVlabs/RVT","repo_kind":"official","path":"rvt/mvt/augmentation.py","file_url":"https://github.com/NVlabs/RVT/blob/HEAD/rvt/mvt/augmentation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"09dc9e73dc9e6c80"}},{"code_sha256_prefix":"0c63dffe9fabe5b9","entry":"place_pc_in_cube","repo":"NVlabs/RVT","repo_kind":"official","path":"rvt/mvt/utils.py","file_url":"https://github.com/NVlabs/RVT/blob/HEAD/rvt/mvt/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"0c63dffe9fabe5b9"}},{"code_sha256_prefix":"ec79cd9a87f20318","entry":"rand_discrete","repo":"NVlabs/RVT","repo_kind":"official","path":"rvt/mvt/aug_utils.py","file_url":"https://github.com/NVlabs/RVT/blob/HEAD/rvt/mvt/aug_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"ec79cd9a87f20318"}},{"code_sha256_prefix":"426bb232645ec17f","entry":"rand_dist","repo":"NVlabs/RVT","repo_kind":"official","path":"rvt/mvt/aug_utils.py","file_url":"https://github.com/NVlabs/RVT/blob/HEAD/rvt/mvt/aug_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"426bb232645ec17f"}},{"code_sha256_prefix":"ffa1b9799883b91f","entry":"select_feat_from_hm","repo":"NVlabs/RVT","repo_kind":"official","path":"rvt/mvt/renderer.py","file_url":"https://github.com/NVlabs/RVT/blob/HEAD/rvt/mvt/renderer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"ffa1b9799883b91f"}},{"code_sha256_prefix":"bdc47824aee91b47","entry":"trans_pc","repo":"NVlabs/RVT","repo_kind":"official","path":"rvt/mvt/utils.py","file_url":"https://github.com/NVlabs/RVT/blob/HEAD/rvt/mvt/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"bdc47824aee91b47"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}