{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/spatio-temporal-lstm-with-trust-gates-for-3d","title":"Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition","arxiv_id":"1607.07043","date":"2016-07-24","proceeding":null,"authors":["Jun Liu","Amir Shahroudy","Dong Xu","Gang Wang"],"abstract":"3D action recognition - analysis of human actions based on 3D skeleton data -\nbecomes popular recently due to its succinctness, robustness, and\nview-invariant representation. Recent attempts on this problem suggested to\ndevelop RNN-based learning methods to model the contextual dependency in the\ntemporal domain. In this paper, we extend this idea to spatio-temporal domains\nto analyze the hidden sources of action-related information within the input\ndata over both domains concurrently. Inspired by the graphical structure of the\nhuman skeleton, we further propose a more powerful tree-structure based\ntraversal method. To handle the noise and occlusion in 3D skeleton data, we\nintroduce new gating mechanism within LSTM to learn the reliability of the\nsequential input data and accordingly adjust its effect on updating the\nlong-term context information stored in the memory cell. Our method achieves\nstate-of-the-art performance on 4 challenging benchmark datasets for 3D human\naction analysis.","url_abs":"http://arxiv.org/abs/1607.07043v1","url_pdf":"http://arxiv.org/pdf/1607.07043v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"3d-human-action-recognition","task_name":"3D Action Recognition"},{"task_slug":"action-analysis","task_name":"Action Analysis"},{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"skeleton-based-action-recognition","task_name":"Skeleton Based Action Recognition"},{"task_slug":"action-recognition","task_name":"Temporal Action Localization"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/skeleton-based-action-recognition-on-ntu-rgbd","task":"Skeleton Based Action Recognition","dataset":"NTU RGB+D","model":"Spatio-Temporal LSTM","rank_in_archive_order":127,"of":135,"metrics":{"Accuracy (CS)":"69.2","Accuracy (CV)":"77.7"},"uses_additional_data":false},{"leaderboard":"/sota/skeleton-based-action-recognition-on-ntu-rgbd","task":"Skeleton Based Action Recognition","dataset":"NTU RGB+D","model":"ST-LSTM","rank_in_archive_order":129,"of":135,"metrics":{"Accuracy (CS)":"61.70","Accuracy (CV)":"75.50"},"uses_additional_data":false},{"leaderboard":"/sota/skeleton-based-action-recognition-on-ntu-rgbd-1","task":"Skeleton Based Action Recognition","dataset":"NTU RGB+D 120","model":"Spatio-Temporal LSTM","rank_in_archive_order":80,"of":83,"metrics":{"Accuracy (Cross-Setup)":"57.9%","Accuracy (Cross-Subject)":"55.7%"},"uses_additional_data":false},{"leaderboard":"/sota/skeleton-based-action-recognition-on-sbu","task":"Skeleton Based Action Recognition","dataset":"SBU / SBU-Refine","model":"ST-LSTM + Trust Gate","rank_in_archive_order":9,"of":9,"metrics":{"Accuracy":"93.3%"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1607.07043","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}