{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cnn-rnn-depth-and-skeleton-based-dynamic-hand","title":"CNN+RNN Depth and Skeleton based Dynamic Hand Gesture Recognition","arxiv_id":"2007.11983","date":"2020-07-22","proceeding":null,"authors":["Kenneth Lai","Svetlana N. Yanushkevich"],"abstract":"Human activity and gesture recognition is an important component of rapidly growing domain of ambient intelligence, in particular in assisting living and smart homes. In this paper, we propose to combine the power of two deep learning techniques, the convolutional neural networks (CNN) and the recurrent neural networks (RNN), for automated hand gesture recognition using both depth and skeleton data. Each of these types of data can be used separately to train neural networks to recognize hand gestures. While RNN were reported previously to perform well in recognition of sequences of movement for each skeleton joint given the skeleton information only, this study aims at utilizing depth data and apply CNN to extract important spatial information from the depth images. Together, the tandem CNN+RNN is capable of recognizing a sequence of gestures more accurately. As well, various types of fusion are studied to combine both the skeleton and depth information in order to extract temporal-spatial information. An overall accuracy of 85.46% is achieved on the dynamic hand gesture-14/28 dataset.","url_abs":"https://arxiv.org/abs/2007.11983v1","url_pdf":"https://arxiv.org/pdf/2007.11983v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"gesture-recognition","task_name":"Gesture Recognition"},{"task_slug":"hand-gesture-recognition","task_name":"Hand Gesture Recognition"},{"task_slug":"hand-gesture-recognition-1","task_name":"Hand-Gesture Recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/hand-gesture-recognition-on-dhg-14","task":"Hand Gesture Recognition","dataset":"DHG-14","model":"SL-fusion-Average","rank_in_archive_order":10,"of":13,"metrics":{"Accuracy":"85.46"},"uses_additional_data":false},{"leaderboard":"/sota/hand-gesture-recognition-on-dhg-14","task":"Hand Gesture Recognition","dataset":"DHG-14","model":"SL-fusion-Maximum","rank_in_archive_order":11,"of":13,"metrics":{"Accuracy":"85.36"},"uses_additional_data":false},{"leaderboard":"/sota/hand-gesture-recognition-on-dhg-14","task":"Hand Gesture Recognition","dataset":"DHG-14","model":"FL-fusion-Concat","rank_in_archive_order":13,"of":13,"metrics":{"Accuracy":"81.86"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}