{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/capturing-complex-hand-movements-and-object","title":"Capturing complex hand movements and object interactions using machine learning-powered stretchable smart textile gloves","arxiv_id":"2410.02221","date":"2024-10-03","proceeding":null,"authors":["Arvin Tashakori","Zenan Jiang","Amir Servati","Saeid Soltanian","Harishkumar Narayana","Katherine Le","Caroline Nakayama","Chieh-ling Yang","Z. Jane Wang","Janice J. Eng","Peyman Servati"],"abstract":"Accurate real-time tracking of dexterous hand movements and interactions has numerous applications in human-computer interaction, metaverse, robotics, and tele-health. Capturing realistic hand movements is challenging because of the large number of articulations and degrees of freedom. Here, we report accurate and dynamic tracking of articulated hand and finger movements using stretchable, washable smart gloves with embedded helical sensor yarns and inertial measurement units. The sensor yarns have a high dynamic range, responding to low 0.005 % to high 155 % strains, and show stability during extensive use and washing cycles. We use multi-stage machine learning to report average joint angle estimation root mean square errors of 1.21 and 1.45 degrees for intra- and inter-subjects cross-validation, respectively, matching accuracy of costly motion capture cameras without occlusion or field of view limitations. We report a data augmentation technique that enhances robustness to noise and variations of sensors. We demonstrate accurate tracking of dexterous hand movements during object interactions, opening new avenues of applications including accurate typing on a mock paper keyboard, recognition of complex dynamic and static gestures adapted from American Sign Language and object identification.","url_abs":"https://arxiv.org/abs/2410.02221v1","url_pdf":"https://arxiv.org/pdf/2410.02221v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"capturing-complex-hand-movements-and-object","repo_url":"https://github.com/arvintashakori/SmartTextileGlove","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"GPL-3.0"}}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"}],"methods":[{"method_slug":null,"method_name":"American"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.02221","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.02221"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/arvintashakori/SmartTextileGlove","reach":{"status":"ok","spdx":"GPL-3.0"}}],"summary":{"ran":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"b1e1f73b5044b74e","entry":"create_inout_sequences","repo":"arvintashakori/SmartTextileGlove","repo_kind":"official","path":"Codes/Python/Script/utils.py","file_url":"https://github.com/arvintashakori/SmartTextileGlove/blob/HEAD/Codes/Python/Script/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"b1e1f73b5044b74e"}},{"code_sha256_prefix":"78d5a8e385b2ce7c","entry":"r2_loss","repo":"arvintashakori/SmartTextileGlove","repo_kind":"official","path":"Codes/Python/Script/utils.py","file_url":"https://github.com/arvintashakori/SmartTextileGlove/blob/HEAD/Codes/Python/Script/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"78d5a8e385b2ce7c"}},{"code_sha256_prefix":"0e99a8e941891536","entry":"rmse_loss","repo":"arvintashakori/SmartTextileGlove","repo_kind":"official","path":"Codes/Python/Script/utils.py","file_url":"https://github.com/arvintashakori/SmartTextileGlove/blob/HEAD/Codes/Python/Script/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"0e99a8e941891536"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}