{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learn-to-cycle-time-consistent-feature","title":"Learn to cycle: Time-consistent feature discovery for action recognition","arxiv_id":"2006.08247","date":"2020-06-15","proceeding":null,"authors":["Alexandros Stergiou","Ronald Poppe"],"abstract":"Generalizing over temporal variations is a prerequisite for effective action recognition in videos. Despite significant advances in deep neural networks, it remains a challenge to focus on short-term discriminative motions in relation to the overall performance of an action. We address this challenge by allowing some flexibility in discovering relevant spatio-temporal features. We introduce Squeeze and Recursion Temporal Gates (SRTG), an approach that favors inputs with similar activations with potential temporal variations. We implement this idea with a novel CNN block that uses an LSTM to encapsulate feature dynamics, in conjunction with a temporal gate that is responsible for evaluating the consistency of the discovered dynamics and the modeled features. We show consistent improvement when using SRTG blocks, with only a minimal increase in the number of GFLOPs. On Kinetics-700, we perform on par with current state-of-the-art models, and outperform these on HACS, Moments in Time, UCF-101 and HMDB-51.","url_abs":"https://arxiv.org/abs/2006.08247v2","url_pdf":"https://arxiv.org/pdf/2006.08247v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learn-to-cycle-time-consistent-feature","repo_url":"https://github.com/alexandrosstergiou/Squeeze-and-Recursion-Temporal-Gates/blob/master/train_model.py","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"action-classification","task_name":"Action Classification"},{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"action-recognition-in-videos-2","task_name":"Action Recognition In Videos"},{"task_slug":"video-classification","task_name":"Video Classification"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/action-classification-on-kinetics-700","task":"Action Classification","dataset":"Kinetics-700","model":"SRTG r3d-101","rank_in_archive_order":31,"of":36,"metrics":{"Top-1 Accuracy":"56.46","Top-5 Accuracy":"76.82"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-kinetics-700","task":"Action Classification","dataset":"Kinetics-700","model":"SRTG r(2+1)d-50","rank_in_archive_order":32,"of":36,"metrics":{"Top-1 Accuracy":"54.17","Top-5 Accuracy":"74.62"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-kinetics-700","task":"Action Classification","dataset":"Kinetics-700","model":"SRTG r3d-50","rank_in_archive_order":33,"of":36,"metrics":{"Top-1 Accuracy":"53.52","Top-5 Accuracy":"74.17"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-kinetics-700","task":"Action Classification","dataset":"Kinetics-700","model":"SRTG r(2+1)d-34","rank_in_archive_order":35,"of":36,"metrics":{"Top-1 Accuracy":"49.43","Top-5 Accuracy":"73.23"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-kinetics-700","task":"Action Classification","dataset":"Kinetics-700","model":"SRTG r3d-34","rank_in_archive_order":36,"of":36,"metrics":{"Top-1 Accuracy":"49.15","Top-5 Accuracy":"72.68"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-moments-in-time","task":"Action Classification","dataset":"MiT","model":"SRTG r3d-101","rank_in_archive_order":17,"of":29,"metrics":{"Top 1 Accuracy":"33.56","Top 5 Accuracy":"58.49"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-moments-in-time","task":"Action Classification","dataset":"MiT","model":"SRTG r(2+1)d-50","rank_in_archive_order":21,"of":29,"metrics":{"Top 1 Accuracy":"31.60","Top 5 Accuracy":"56.80"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-moments-in-time","task":"Action Classification","dataset":"MiT","model":"SRTG r3d-50","rank_in_archive_order":22,"of":29,"metrics":{"Top 1 Accuracy":"30.72","Top 5 Accuracy":"55.65"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-moments-in-time","task":"Action Classification","dataset":"MiT","model":"SRTG r(2+1)d-34","rank_in_archive_order":24,"of":29,"metrics":{"Top 1 Accuracy":"28.97","Top 5 Accuracy":"54.18"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-moments-in-time","task":"Action Classification","dataset":"MiT","model":"SRTG r3d-34","rank_in_archive_order":25,"of":29,"metrics":{"Top 1 Accuracy":"28.55","Top 5 Accuracy":"52.35"},"uses_additional_data":false},{"leaderboard":"/sota/action-recognition-on-hacs","task":"Action Recognition","dataset":"HACS","model":"SRTG r(2+1)d-101","rank_in_archive_order":3,"of":8,"metrics":{"Top 1 Accuracy":"84.33","Top 5 Accuracy":"96.85"},"uses_additional_data":false},{"leaderboard":"/sota/action-recognition-on-hacs","task":"Action Recognition","dataset":"HACS","model":"SRTG r(2+1)d-50","rank_in_archive_order":4,"of":8,"metrics":{"Top 1 Accuracy":"83.77","Top 5 Accuracy":"96.56"},"uses_additional_data":false},{"leaderboard":"/sota/action-recognition-on-hacs","task":"Action Recognition","dataset":"HACS","model":"SRTG r3d-101","rank_in_archive_order":5,"of":8,"metrics":{"Top 1 Accuracy":"81.66","Top 5 Accuracy":"96.33"},"uses_additional_data":false},{"leaderboard":"/sota/action-recognition-on-hacs","task":"Action Recognition","dataset":"HACS","model":"SRTG r(2+1)d-34","rank_in_archive_order":6,"of":8,"metrics":{"Top 1 Accuracy":"80.39","Top 5 Accuracy":"94.27"},"uses_additional_data":false},{"leaderboard":"/sota/action-recognition-on-hacs","task":"Action Recognition","dataset":"HACS","model":"SRTG r3d-50","rank_in_archive_order":7,"of":8,"metrics":{"Top 1 Accuracy":"80.36","Top 5 Accuracy":"95.55"},"uses_additional_data":false},{"leaderboard":"/sota/action-recognition-on-hacs","task":"Action Recognition","dataset":"HACS","model":"SRTG r3d-34","rank_in_archive_order":8,"of":8,"metrics":{"Top 1 Accuracy":"78.60","Top 5 Accuracy":"93.57"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2006.08247","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}