{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/tada-temporally-adaptive-convolutions-for-1","title":"TAda! Temporally-Adaptive Convolutions for Video Understanding","arxiv_id":"2110.06178","date":"2021-10-12","proceeding":"ICLR 2022 4","authors":["Ziyuan Huang","Shiwei Zhang","Liang Pan","Zhiwu Qing","Mingqian Tang","Ziwei Liu","Marcelo H. Ang Jr"],"abstract":"Spatial convolutions are widely used in numerous deep video models. It fundamentally assumes spatio-temporal invariance, i.e., using shared weights for every location in different frames. This work presents Temporally-Adaptive Convolutions (TAdaConv) for video understanding, which shows that adaptive weight calibration along the temporal dimension is an efficient way to facilitate modelling complex temporal dynamics in videos. Specifically, TAdaConv empowers the spatial convolutions with temporal modelling abilities by calibrating the convolution weights for each frame according to its local and global temporal context. Compared to previous temporal modelling operations, TAdaConv is more efficient as it operates over the convolution kernels instead of the features, whose dimension is an order of magnitude smaller than the spatial resolutions. Further, the kernel calibration brings an increased model capacity. We construct TAda2D and TAdaConvNeXt networks by replacing the 2D convolutions in ResNet and ConvNeXt with TAdaConv, which leads to at least on par or better performance compared to state-of-the-art approaches on multiple video action recognition and localization benchmarks. We also demonstrate that as a readily plug-in operation with negligible computation overhead, TAdaConv can effectively improve many existing video models with a convincing margin.","url_abs":"https://arxiv.org/abs/2110.06178v4","url_pdf":"https://arxiv.org/pdf/2110.06178v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"tada-temporally-adaptive-convolutions-for-1","repo_url":"https://github.com/alibaba-mmai-research/pytorch-video-understanding","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"tada-temporally-adaptive-convolutions-for-1","repo_url":"https://github.com/alibaba-mmai-research/TAdaConv","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"action-classification","task_name":"Action Classification"},{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"action-recognition","task_name":"Temporal Action Localization"},{"task_slug":"video-understanding","task_name":"Video Understanding"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/action-classification-on-kinetics-400","task":"Action Classification","dataset":"Kinetics-400","model":"TAdaConvNeXt-T","rank_in_archive_order":120,"of":207,"metrics":{"Acc@1":"79.1","Acc@5":"93.7"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-kinetics-400","task":"Action Classification","dataset":"Kinetics-400","model":"TAda2D-En (ResNet-50, 8+16 frames)","rank_in_archive_order":128,"of":207,"metrics":{"Acc@1":"78.2","Acc@5":"93.5"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-kinetics-400","task":"Action Classification","dataset":"Kinetics-400","model":"TAda2D (ResNet-50, 16 frames)","rank_in_archive_order":139,"of":207,"metrics":{"Acc@1":"77.4","Acc@5":"93.1"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-kinetics-400","task":"Action Classification","dataset":"Kinetics-400","model":"TAda2D (ResNet-50, 8 frames)","rank_in_archive_order":146,"of":207,"metrics":{"Acc@1":"76.7","Acc@5":"92.6"},"uses_additional_data":false},{"leaderboard":"/sota/action-recognition-in-videos-on-something","task":"Action Recognition","dataset":"Something-Something V2","model":"TAda2D-En (ResNet-50, 8+16 frames)","rank_in_archive_order":69,"of":123,"metrics":{"Top-1 Accuracy":"67.2","Top-5 Accuracy":"89.8"},"uses_additional_data":true},{"leaderboard":"/sota/action-recognition-in-videos-on-something","task":"Action Recognition","dataset":"Something-Something V2","model":"TAdaConvNeXt-T","rank_in_archive_order":71,"of":123,"metrics":{"Top-1 Accuracy":"67.1","Top-5 Accuracy":"90.4"},"uses_additional_data":true},{"leaderboard":"/sota/action-recognition-in-videos-on-something","task":"Action Recognition","dataset":"Something-Something V2","model":"TAda2D (ResNet-50, 16 frames)","rank_in_archive_order":87,"of":123,"metrics":{"Top-1 Accuracy":"65.6","Top-5 Accuracy":"89.2"},"uses_additional_data":true},{"leaderboard":"/sota/action-recognition-in-videos-on-something","task":"Action Recognition","dataset":"Something-Something V2","model":"TAda2D (ResNet-50, 8 frames)","rank_in_archive_order":97,"of":123,"metrics":{"Top-1 Accuracy":"64.0","Top-5 Accuracy":"88.0"},"uses_additional_data":true}],"syntology":{"syntology_url":"https://syntology.ai/paper/2110.06178","atlas_url":"https://app.syntology.ai/?focus=2110.06178","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.06178"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/alibaba-mmai-research/TAdaConv","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/alibaba-mmai-research/pytorch-video-understanding","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":1},"by_repo_kind":{"listed":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"cf623eebda41b1bf","entry":"TAdaConv2d","repo":"alibaba-mmai-research/TAdaConv","repo_kind":"listed","path":"tadaconv/models/module_zoo/ops/tadaconv.py","file_url":"https://github.com/alibaba-mmai-research/TAdaConv/blob/HEAD/tadaconv/models/module_zoo/ops/tadaconv.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cf623eebda41b1bf"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}