{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/grasp-as-you-say-language-guided-dexterous","title":"Grasp as You Say: Language-guided Dexterous Grasp Generation","arxiv_id":"2405.19291","date":"2024-05-29","proceeding":null,"authors":["Yi-Lin Wei","Jian-Jian Jiang","Chengyi Xing","Xian-Tuo Tan","Xiao-Ming Wu","Hao Li","Mark Cutkosky","Wei-Shi Zheng"],"abstract":"This paper explores a novel task \"Dexterous Grasp as You Say\" (DexGYS), enabling robots to perform dexterous grasping based on human commands expressed in natural language. However, the development of this field is hindered by the lack of datasets with natural human guidance; thus, we propose a language-guided dexterous grasp dataset, named DexGYSNet, offering high-quality dexterous grasp annotations along with flexible and fine-grained human language guidance. Our dataset construction is cost-efficient, with the carefully-design hand-object interaction retargeting strategy, and the LLM-assisted language guidance annotation system. Equipped with this dataset, we introduce the DexGYSGrasp framework for generating dexterous grasps based on human language instructions, with the capability of producing grasps that are intent-aligned, high quality and diversity. To achieve this capability, our framework decomposes the complex learning process into two manageable progressive objectives and introduce two components to realize them. The first component learns the grasp distribution focusing on intention alignment and generation diversity. And the second component refines the grasp quality while maintaining intention consistency. Extensive experiments are conducted on DexGYSNet and real world environments for validation.","url_abs":"https://arxiv.org/abs/2405.19291v2","url_pdf":"https://arxiv.org/pdf/2405.19291v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"grasp-as-you-say-language-guided-dexterous","repo_url":"https://github.com/iSEE-Laboratory/Grasp-as-You-Say","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2405.19291","atlas_url":"https://app.syntology.ai/?focus=2405.19291","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.19291"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/iSEE-Laboratory/Grasp-as-You-Say","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"ran_honours":1,"ran":3,"ran_fixture":1,"unverified":1},"by_repo_kind":{"official":{"samples":7,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"fac5364e2f53c6db","entry":"conv3x3","repo":"iSEE-Laboratory/Grasp-as-You-Say","repo_kind":"official","path":"model/backbone/resnet.py","file_url":"https://github.com/iSEE-Laboratory/Grasp-as-You-Say/blob/HEAD/model/backbone/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fac5364e2f53c6db"}},{"code_sha256_prefix":"ec413739d406e611","entry":"pc_normalize","repo":"iSEE-Laboratory/Grasp-as-You-Say","repo_kind":"official","path":"model/backbone/pointnet2_utils.py","file_url":"https://github.com/iSEE-Laboratory/Grasp-as-You-Say/blob/HEAD/model/backbone/pointnet2_utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ec413739d406e611"}},{"code_sha256_prefix":"abfa45e99be2f608","entry":"resnet18","repo":"iSEE-Laboratory/Grasp-as-You-Say","repo_kind":"official","path":"model/backbone/resnet.py","file_url":"https://github.com/iSEE-Laboratory/Grasp-as-You-Say/blob/HEAD/model/backbone/resnet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"abfa45e99be2f608"}},{"code_sha256_prefix":"05a69c79378c5973","entry":"resnet34","repo":"iSEE-Laboratory/Grasp-as-You-Say","repo_kind":"official","path":"model/backbone/resnet.py","file_url":"https://github.com/iSEE-Laboratory/Grasp-as-You-Say/blob/HEAD/model/backbone/resnet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"05a69c79378c5973"}},{"code_sha256_prefix":"e12b87495d1edcd1","entry":"size_splits","repo":"iSEE-Laboratory/Grasp-as-You-Say","repo_kind":"official","path":"model/backbone/pointnet.py","file_url":"https://github.com/iSEE-Laboratory/Grasp-as-You-Say/blob/HEAD/model/backbone/pointnet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e12b87495d1edcd1"}},{"code_sha256_prefix":"bba5dc7caf857cd2","entry":"square_distance","repo":"iSEE-Laboratory/Grasp-as-You-Say","repo_kind":"official","path":"model/backbone/pointnet2_utils.py","file_url":"https://github.com/iSEE-Laboratory/Grasp-as-You-Say/blob/HEAD/model/backbone/pointnet2_utils.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bba5dc7caf857cd2"}},{"code_sha256_prefix":"b1227ddb721e2999","entry":"timeit","repo":"iSEE-Laboratory/Grasp-as-You-Say","repo_kind":"official","path":"model/backbone/pointnet2_utils.py","file_url":"https://github.com/iSEE-Laboratory/Grasp-as-You-Say/blob/HEAD/model/backbone/pointnet2_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b1227ddb721e2999"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}