{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gaussiangrasper-3d-language-gaussian","title":"GaussianGrasper: 3D Language Gaussian Splatting for Open-vocabulary Robotic Grasping","arxiv_id":"2403.09637","date":"2024-03-14","proceeding":null,"authors":["Yuhang Zheng","Xiangyu Chen","Yupeng Zheng","Songen Gu","Runyi Yang","Bu Jin","Pengfei Li","Chengliang Zhong","Zengmao Wang","Lina Liu","Chao Yang","Dawei Wang","Zhen Chen","Xiaoxiao Long","Meiqing Wang"],"abstract":"Constructing a 3D scene capable of accommodating open-ended language queries, is a pivotal pursuit, particularly within the domain of robotics. Such technology facilitates robots in executing object manipulations based on human language directives. To tackle this challenge, some research efforts have been dedicated to the development of language-embedded implicit fields. However, implicit fields (e.g. NeRF) encounter limitations due to the necessity of processing a large number of input views for reconstruction, coupled with their inherent inefficiencies in inference. Thus, we present the GaussianGrasper, which utilizes 3D Gaussian Splatting to explicitly represent the scene as a collection of Gaussian primitives. Our approach takes a limited set of RGB-D views and employs a tile-based splatting technique to create a feature field. In particular, we propose an Efficient Feature Distillation (EFD) module that employs contrastive learning to efficiently and accurately distill language embeddings derived from foundational models. With the reconstructed geometry of the Gaussian field, our method enables the pre-trained grasping model to generate collision-free grasp pose candidates. Furthermore, we propose a normal-guided grasp module to select the best grasp pose. Through comprehensive real-world experiments, we demonstrate that GaussianGrasper enables robots to accurately query and grasp objects with language instructions, providing a new solution for language-guided manipulation tasks. Data and codes can be available at https://github.com/MrSecant/GaussianGrasper.","url_abs":"https://arxiv.org/abs/2403.09637v1","url_pdf":"https://arxiv.org/pdf/2403.09637v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"gaussiangrasper-3d-language-gaussian","repo_url":"https://github.com/mrsecant/gaussiangrasper","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok"}}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"nerf","task_name":"NeRF"},{"task_slug":"robotic-grasping","task_name":"Robotic Grasping"},{"task_slug":"simulated-gaussian-manipulation","task_name":"Simulated Gaussian Manipulation"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2403.09637","atlas_url":"https://app.syntology.ai/?focus=2403.09637","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.09637"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mrsecant/gaussiangrasper","reach":{"status":"ok"}}],"summary":{"ran_honours":2,"ran":6},"by_repo_kind":{"official":{"samples":8,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"3d2c1a432b1a3561","entry":"RGB2SH","repo":"mrsecant/gaussiangrasper","repo_kind":"official","path":"nerfstudio/models/gaussian_splatting.py","file_url":"https://github.com/mrsecant/gaussiangrasper/blob/HEAD/nerfstudio/models/gaussian_splatting.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3d2c1a432b1a3561"}},{"code_sha256_prefix":"508a98451f5b78fe","entry":"SH2RGB","repo":"mrsecant/gaussiangrasper","repo_kind":"official","path":"nerfstudio/models/gaussian_splatting.py","file_url":"https://github.com/mrsecant/gaussiangrasper/blob/HEAD/nerfstudio/models/gaussian_splatting.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"508a98451f5b78fe"}},{"code_sha256_prefix":"f1e7d68796e89ab2","entry":"lossfun_outer","repo":"mrsecant/gaussiangrasper","repo_kind":"official","path":"nerfstudio/model_components/losses.py","file_url":"https://github.com/mrsecant/gaussiangrasper/blob/HEAD/nerfstudio/model_components/losses.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f1e7d68796e89ab2"}},{"code_sha256_prefix":"c64c9a3a66a1d0c8","entry":"merge_methods","repo":"mrsecant/gaussiangrasper","repo_kind":"official","path":"nerfstudio/configs/method_configs.py","file_url":"https://github.com/mrsecant/gaussiangrasper/blob/HEAD/nerfstudio/configs/method_configs.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c64c9a3a66a1d0c8"}},{"code_sha256_prefix":"de818de3a5e2a784","entry":"outer","repo":"mrsecant/gaussiangrasper","repo_kind":"official","path":"nerfstudio/model_components/losses.py","file_url":"https://github.com/mrsecant/gaussiangrasper/blob/HEAD/nerfstudio/model_components/losses.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"de818de3a5e2a784"}},{"code_sha256_prefix":"af506abe9226350c","entry":"random_quat_tensor","repo":"mrsecant/gaussiangrasper","repo_kind":"official","path":"nerfstudio/models/gaussian_splatting.py","file_url":"https://github.com/mrsecant/gaussiangrasper/blob/HEAD/nerfstudio/models/gaussian_splatting.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"af506abe9226350c"}},{"code_sha256_prefix":"34a1673bf755a59b","entry":"ray_samples_to_sdist","repo":"mrsecant/gaussiangrasper","repo_kind":"official","path":"nerfstudio/model_components/losses.py","file_url":"https://github.com/mrsecant/gaussiangrasper/blob/HEAD/nerfstudio/model_components/losses.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"34a1673bf755a59b"}},{"code_sha256_prefix":"06036a2535dc9b18","entry":"sort_methods","repo":"mrsecant/gaussiangrasper","repo_kind":"official","path":"nerfstudio/configs/method_configs.py","file_url":"https://github.com/mrsecant/gaussiangrasper/blob/HEAD/nerfstudio/configs/method_configs.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"06036a2535dc9b18"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}