{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/hierarchical-long-short-term-concurrent","title":"Hierarchical Long Short-Term Concurrent Memory for Human Interaction Recognition","arxiv_id":"1811.00270","date":"2018-11-01","proceeding":null,"authors":["Xiangbo Shu","Jinhui Tang","Guo-Jun Qi","Wei Liu","Jian Yang"],"abstract":"In this paper, we aim to address the problem of human interaction recognition\nin videos by exploring the long-term inter-related dynamics among multiple\npersons. Recently, Long Short-Term Memory (LSTM) has become a popular choice to\nmodel individual dynamic for single-person action recognition due to its\nability of capturing the temporal motion information in a range. However,\nexisting RNN models focus only on capturing the dynamics of human interaction\nby simply combining all dynamics of individuals or modeling them as a whole.\nSuch models neglect the inter-related dynamics of how human interactions change\nover time. To this end, we propose a novel Hierarchical Long Short-Term\nConcurrent Memory (H-LSTCM) to model the long-term inter-related dynamics among\na group of persons for recognizing the human interactions. Specifically, we\nfirst feed each person's static features into a Single-Person LSTM to learn the\nsingle-person dynamic. Subsequently, the outputs of all Single-Person LSTM\nunits are fed into a novel Concurrent LSTM (Co-LSTM) unit, which mainly\nconsists of multiple sub-memory units, a new cell gate and a new co-memory\ncell. In a Co-LSTM unit, each sub-memory unit stores individual motion\ninformation, while this Co-LSTM unit selectively integrates and stores\ninter-related motion information between multiple interacting persons from\nmultiple sub-memory units via the cell gate and co-memory cell, respectively.\nExtensive experiments on four public datasets validate the effectiveness of the\nproposed H-LSTCM by comparing against baseline and state-of-the-art methods.","url_abs":"http://arxiv.org/abs/1811.00270v1","url_pdf":"http://arxiv.org/pdf/1811.00270v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"human-interaction-recognition","task_name":"Human Interaction Recognition"},{"task_slug":"action-recognition","task_name":"Temporal Action Localization"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/group-activity-recognition-on-collective","task":"Group Activity Recognition","dataset":"Collective Activity","model":"H-LSTCM","rank_in_archive_order":5,"of":6,"metrics":{"Accuracy":"83.75"},"uses_additional_data":false},{"leaderboard":"/sota/group-activity-recognition-on-volleyball","task":"Group Activity Recognition","dataset":"Volleyball","model":"H-LSTCM","rank_in_archive_order":11,"of":12,"metrics":{"Accuracy":"88.4"},"uses_additional_data":false},{"leaderboard":"/sota/human-interaction-recognition-on-bit","task":"Human Interaction Recognition","dataset":"BIT","model":"H-LSTCM","rank_in_archive_order":1,"of":3,"metrics":{"Accuracy":"94.03"},"uses_additional_data":false},{"leaderboard":"/sota/human-interaction-recognition-on-ut","task":"Human Interaction Recognition","dataset":"UT","model":"H-LSTCM","rank_in_archive_order":1,"of":4,"metrics":{"Accuracy":"98.33"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1811.00270","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}