{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/skeleton-based-gesture-recognition-using","title":"Skeleton-based Gesture Recognition Using Several Fully Connected Layers with Path Signature Features and Temporal Transformer Module","arxiv_id":"1811.07081","date":"2018-11-17","proceeding":null,"authors":["Chenyang Li","Xin Zhang","Lufan Liao","Lianwen Jin","Weixin Yang"],"abstract":"The skeleton based gesture recognition is gaining more popularity due to its\nwide possible applications. The key issues are how to extract discriminative\nfeatures and how to design the classification model. In this paper, we first\nleverage a robust feature descriptor, path signature (PS), and propose three PS\nfeatures to explicitly represent the spatial and temporal motion\ncharacteristics, i.e., spatial PS (S_PS), temporal PS (T_PS) and temporal\nspatial PS (T_S_PS). Considering the significance of fine hand movements in the\ngesture, we propose an \"attention on hand\" (AOH) principle to define joint\npairs for the S_PS and select single joint for the T_PS. In addition, the\ndyadic method is employed to extract the T_PS and T_S_PS features that encode\nglobal and local temporal dynamics in the motion. Secondly, without the\nrecurrent strategy, the classification model still faces challenges on temporal\nvariation among different sequences. We propose a new temporal transformer\nmodule (TTM) that can match the sequence key frames by learning the temporal\nshifting parameter for each input. This is a learning-based module that can be\nincluded into standard neural network architecture. Finally, we design a\nmulti-stream fully connected layer based network to treat spatial and temporal\nfeatures separately and fused them together for the final result. We have\ntested our method on three benchmark gesture datasets, i.e., ChaLearn 2016,\nChaLearn 2013 and MSRC-12. Experimental results demonstrate that we achieve the\nstate-of-the-art performance on skeleton-based gesture recognition with high\ncomputational efficiency.","url_abs":"http://arxiv.org/abs/1811.07081v2","url_pdf":"http://arxiv.org/pdf/1811.07081v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"skeleton-based-gesture-recognition-using","repo_url":"https://github.com/LiChenyang-Github/Temporal-Transformer-Module","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"computational-efficiency","task_name":"Computational Efficiency"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"gesture-recognition","task_name":"Gesture Recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/gesture-recognition-on-chalearn-2013","task":"Gesture Recognition","dataset":"ChaLearn 2013","model":"3S Net TTM","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"92.08"},"uses_additional_data":false},{"leaderboard":"/sota/gesture-recognition-on-chalearn-2016","task":"Gesture Recognition","dataset":"ChaLearn 2016","model":"3S Net TTM","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"39.95"},"uses_additional_data":false},{"leaderboard":"/sota/gesture-recognition-on-msrc-12","task":"Gesture Recognition","dataset":"MSRC-12","model":"3S Net TTM","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"99.01"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1811.07081","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}