{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/interaction-relational-network-for-mutual","title":"Interaction Relational Network for Mutual Action Recognition","arxiv_id":"1910.04963","date":"2019-10-11","proceeding":null,"authors":["Mauricio Perez","Jun Liu","Alex C. Kot"],"abstract":"Person-person mutual action recognition (also referred to as interaction recognition) is an important research branch of human activity analysis. Current solutions in the field -- mainly dominated by CNNs, GCNs and LSTMs -- often consist of complicated architectures and mechanisms to embed the relationships between the two persons on the architecture itself, to ensure the interaction patterns can be properly learned. Our main contribution with this work is by proposing a simpler yet very powerful architecture, named Interaction Relational Network, which utilizes minimal prior knowledge about the structure of the human body. We drive the network to identify by itself how to relate the body parts from the individuals interacting. In order to better represent the interaction, we define two different relationships, leading to specialized architectures and models for each. These multiple relationship models will then be fused into a single and special architecture, in order to leverage both streams of information for further enhancing the relational reasoning capability. Furthermore we define important structured pair-wise operations to extract meaningful extra information from each pair of joints -- distance and motion. Ultimately, with the coupling of an LSTM, our IRN is capable of paramount sequential relational reasoning. These important extensions we made to our network can also be valuable to other problems that require sophisticated relational reasoning. Our solution is able to achieve state-of-the-art performance on the traditional interaction recognition datasets SBU and UT, and also on the mutual actions from the large-scale dataset NTU RGB+D. Furthermore, it obtains competitive performance in the NTU RGB+D 120 dataset interactions subset.","url_abs":"https://arxiv.org/abs/1910.04963v2","url_pdf":"https://arxiv.org/pdf/1910.04963v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"interaction-relational-network-for-mutual","repo_url":"https://github.com/mauriciolp/inter-rel-net","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"human-interaction-recognition","task_name":"Human Interaction Recognition"},{"task_slug":"relational-reasoning","task_name":"Relational Reasoning"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/human-interaction-recognition-on-ntu-rgb-d","task":"Human Interaction Recognition","dataset":"NTU RGB+D","model":"LSTM-IRN'fc1inter+intra","rank_in_archive_order":5,"of":5,"metrics":{"Accuracy (Cross-Subject)":"90.5","Accuracy (Cross-View)":"93.5"},"uses_additional_data":false},{"leaderboard":"/sota/human-interaction-recognition-on-ntu-rgb-d-1","task":"Human Interaction Recognition","dataset":"NTU RGB+D 120","model":"LSTM-IRN","rank_in_archive_order":6,"of":6,"metrics":{"Accuracy (Cross-Setup)":"79.6","Accuracy (Cross-Subject)":"77.7"},"uses_additional_data":false},{"leaderboard":"/sota/human-interaction-recognition-on-sbu-1","task":"Human Interaction Recognition","dataset":"SBU","model":"LSTM-IRN'fc1inter+intra","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"98.2"},"uses_additional_data":false},{"leaderboard":"/sota/human-interaction-recognition-on-ut-1","task":"Human Interaction Recognition","dataset":"UT-Interaction","model":"LSTM-IRN'fc1inter+intra","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy (Set 1)":"98.3","Accuracy (Set 2)":"96.7"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1910.04963","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1910.04963"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mauriciolp/inter-rel-net","reach":null}],"summary":{"ran_draft_wrong":1,"ran_honours":1},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d2e9220a44032c76","entry":"get_relevant_kwargs","repo":"mauriciolp/inter-rel-net","repo_kind":"official","path":"src/models/rn.py","file_url":"https://github.com/mauriciolp/inter-rel-net/blob/HEAD/src/models/rn.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d2e9220a44032c76"}},{"code_sha256_prefix":"29eba85d6d5181d1","entry":"parse_fit_history","repo":"mauriciolp/inter-rel-net","repo_kind":"official","path":"src/run_protocol.py","file_url":"https://github.com/mauriciolp/inter-rel-net/blob/HEAD/src/run_protocol.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"29eba85d6d5181d1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}