{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/appearance-and-relation-networks-for-video","title":"Appearance-and-Relation Networks for Video Classification","arxiv_id":"1711.09125","date":"2017-11-24","proceeding":"CVPR 2018 6","authors":["Limin Wang","Wei Li","Wen Li","Luc van Gool"],"abstract":"Spatiotemporal feature learning in videos is a fundamental problem in\ncomputer vision. This paper presents a new architecture, termed as\nAppearance-and-Relation Network (ARTNet), to learn video representation in an\nend-to-end manner. ARTNets are constructed by stacking multiple generic\nbuilding blocks, called as SMART, whose goal is to simultaneously model\nappearance and relation from RGB input in a separate and explicit manner.\nSpecifically, SMART blocks decouple the spatiotemporal learning module into an\nappearance branch for spatial modeling and a relation branch for temporal\nmodeling. The appearance branch is implemented based on the linear combination\nof pixels or filter responses in each frame, while the relation branch is\ndesigned based on the multiplicative interactions between pixels or filter\nresponses across multiple frames. We perform experiments on three action\nrecognition benchmarks: Kinetics, UCF101, and HMDB51, demonstrating that SMART\nblocks obtain an evident improvement over 3D convolutions for spatiotemporal\nfeature learning. Under the same training setting, ARTNets achieve superior\nperformance on these three datasets to the existing state-of-the-art methods.","url_abs":"http://arxiv.org/abs/1711.09125v2","url_pdf":"http://arxiv.org/pdf/1711.09125v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"appearance-and-relation-networks-for-video","repo_url":"https://github.com/wanglimin/ARTNet","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"action-classification","task_name":"Action Classification"},{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":null,"task_name":"Relation"},{"task_slug":"relation-network","task_name":"Relation Network"},{"task_slug":"action-recognition","task_name":"Temporal Action Localization"},{"task_slug":"video-classification","task_name":"Video Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/action-classification-on-kinetics-400","task":"Action Classification","dataset":"Kinetics-400","model":"ARTNet","rank_in_archive_order":173,"of":207,"metrics":{"Acc@1":"72.4","Acc@5":"90.4"},"uses_additional_data":false},{"leaderboard":"/sota/action-recognition-in-videos-on-hmdb-51","task":"Action Recognition","dataset":"HMDB-51","model":"ARTNet w/ TSN","rank_in_archive_order":53,"of":77,"metrics":{"Average accuracy of 3 splits":"70.9"},"uses_additional_data":false},{"leaderboard":"/sota/action-recognition-in-videos-on-ucf101","task":"Action Recognition","dataset":"UCF101","model":"ARTNet w/ TSN","rank_in_archive_order":55,"of":91,"metrics":{"3-fold Accuracy":"94.3"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1711.09125","atlas_url":"https://app.syntology.ai/?focus=1711.09125","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}