{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/temporal-3d-convnets-new-architecture-and","title":"Temporal 3D ConvNets: New Architecture and Transfer Learning for Video Classification","arxiv_id":"1711.08200","date":"2017-11-22","proceeding":null,"authors":["Ali Diba","Mohsen Fayyaz","Vivek Sharma","Amir Hossein Karami","Mohammad Mahdi Arzani","Rahman Yousefzadeh","Luc van Gool"],"abstract":"The work in this paper is driven by the question how to exploit the temporal\ncues available in videos for their accurate classification, and for human\naction recognition in particular? Thus far, the vision community has focused on\nspatio-temporal approaches with fixed temporal convolution kernel depths. We\nintroduce a new temporal layer that models variable temporal convolution kernel\ndepths. We embed this new temporal layer in our proposed 3D CNN. We extend the\nDenseNet architecture - which normally is 2D - with 3D filters and pooling\nkernels. We name our proposed video convolutional network `Temporal 3D\nConvNet'~(T3D) and its new temporal layer `Temporal Transition Layer'~(TTL).\nOur experiments show that T3D outperforms the current state-of-the-art methods\non the HMDB51, UCF101 and Kinetics datasets.\n  The other issue in training 3D ConvNets is about training them from scratch\nwith a huge labeled dataset to get a reasonable performance. So the knowledge\nlearned in 2D ConvNets is completely ignored. Another contribution in this work\nis a simple and effective technique to transfer knowledge from a pre-trained 2D\nCNN to a randomly initialized 3D CNN for a stable weight initialization. This\nallows us to significantly reduce the number of training samples for 3D CNNs.\nThus, by finetuning this network, we beat the performance of generic and recent\nmethods in 3D CNNs, which were trained on large video datasets, e.g. Sports-1M,\nand finetuned on the target datasets, e.g. HMDB51/UCF101. The T3D codes will be\nreleased","url_abs":"http://arxiv.org/abs/1711.08200v1","url_pdf":"http://arxiv.org/pdf/1711.08200v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"temporal-3d-convnets-new-architecture-and","repo_url":"https://github.com/MohsenFayyaz89/T3D","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"temporal-3d-convnets-new-architecture-and","repo_url":"https://github.com/rekon/T3D-keras","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"temporal-3d-convnets-new-architecture-and","repo_url":"https://github.com/yfxc/tensorflow-T3D","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"action-recognition","task_name":"Temporal Action Localization"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"},{"task_slug":"video-classification","task_name":"Video Classification"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1711.08200","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}