{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/global-temporal-difference-network-for-action","title":"Global Temporal Difference Network for Action Recognition","arxiv_id":null,"date":"2022-11-23","proceeding":"TMM 2022 11","authors":["Zhao Xie","Jiansong Chen","Kewei Wu","Dan Guo","Richang Hong"],"abstract":"—Temporal modeling still remains as a challenge\r\nfor action recognition. Most existing temporal models focus on\r\nlearning local variation between neighbor frames. There exists\r\nobvious deviations between local and global variations, such as\r\nsubtle and notable motion variations. In this paper, we propose a\r\nglobal temporal difference module for action recognition, which\r\nconsists of two sub-modules, i.e., a global aggregation module and\r\na global difference module. These two sub-modules cooperate\r\nfollowing the idea of using prior knowledge from the global\r\nview (i.e., global motion variation) to guide local learning at\r\neach moment. In the global aggregation module, the global\r\nprior knowledge is learned by aggregating the visual feature\r\nsequence of video into a global vector. In the global difference\r\nmodule, we prepare the difference vector sequence of video\r\nby subtracting each local vector from the global vector. Our\r\nmethod performs as a contextual guidance with a global view.\r\nThe sequential dependency between these difference vectors is\r\nexploited with a channel-wise self-attention operation. Finally,\r\nthe difference vectors at each timestamp are further used to\r\nenhance the semantics of the original local features. The enhanced\r\nfeatures endow the action recognition has less deviation to\r\nunderstand the variation in the video globally. We instantiate\r\nthe global temporal difference module into the ResNet block to\r\nform a global temporal difference network (GTDNet). Exhaustive\r\nexperiments are conducted and our method achieves competitive\r\nperformance at small FLOPs on Something-Something V1 & V2\r\nand Kinetics-400.","url_abs":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9961904","url_pdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9961904","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/action-recognition-in-videos-on-something","task":"Action Recognition","dataset":"Something-Something V2","model":"GTDNet","rank_in_archive_order":63,"of":123,"metrics":{"Top-1 Accuracy":"67.6"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}