{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/would-mega-scale-datasets-further-enhance","title":"Would Mega-scale Datasets Further Enhance Spatiotemporal 3D CNNs?","arxiv_id":"2004.04968","date":"2020-04-10","proceeding":null,"authors":["Hirokatsu Kataoka","Tenga Wakamiya","Kensho Hara","Yutaka Satoh"],"abstract":"How can we collect and use a video dataset to further improve spatiotemporal 3D Convolutional Neural Networks (3D CNNs)? In order to positively answer this open question in video recognition, we have conducted an exploration study using a couple of large-scale video datasets and 3D CNNs. In the early era of deep neural networks, 2D CNNs have been better than 3D CNNs in the context of video recognition. Recent studies revealed that 3D CNNs can outperform 2D CNNs trained on a large-scale video dataset. However, we heavily rely on architecture exploration instead of dataset consideration. Therefore, in the present paper, we conduct exploration study in order to improve spatiotemporal 3D CNNs as follows: (i) Recently proposed large-scale video datasets help improve spatiotemporal 3D CNNs in terms of video classification accuracy. We reveal that a carefully annotated dataset (e.g., Kinetics-700) effectively pre-trains a video representation for a video classification task. (ii) We confirm the relationships between #category/#instance and video classification accuracy. The results show that #category should initially be fixed, and then #instance is increased on a video dataset in case of dataset construction. (iii) In order to practically extend a video dataset, we simply concatenate publicly available datasets, such as Kinetics-700 and Moments in Time (MiT) datasets. Compared with Kinetics-700 pre-training, we further enhance spatiotemporal 3D CNNs with the merged dataset, e.g., +0.9, +3.4, and +1.1 on UCF-101, HMDB-51, and ActivityNet datasets, respectively, in terms of fine-tuning. (iv) In terms of recognition architecture, the Kinetics-700 and merged dataset pre-trained models increase the recognition performance to 200 layers with the Residual Network (ResNet), while the Kinetics-400 pre-trained model cannot successfully optimize the 200-layer architecture.","url_abs":"https://arxiv.org/abs/2004.04968v1","url_pdf":"https://arxiv.org/pdf/2004.04968v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"would-mega-scale-datasets-further-enhance","repo_url":"https://github.com/kenshohara/3D-ResNets-PyTorch","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"would-mega-scale-datasets-further-enhance","repo_url":"https://github.com/3D-ResNets-Reproduction/Reference-Project","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"would-mega-scale-datasets-further-enhance","repo_url":"https://github.com/KevinPang180/3D-ResNets-PaddlePaddle","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"would-mega-scale-datasets-further-enhance","repo_url":"https://github.com/Mind23-2/MindCode-68","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"mindspore","reach":{"status":"ok"}},{"paper_slug":"would-mega-scale-datasets-further-enhance","repo_url":"https://github.com/Oichii/resnet3D_pulse","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"would-mega-scale-datasets-further-enhance","repo_url":"https://github.com/Shito0907/3D_resnet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"would-mega-scale-datasets-further-enhance","repo_url":"https://github.com/kaleab-k/VideoAT","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"would-mega-scale-datasets-further-enhance","repo_url":"https://github.com/rutajoshi/cs231n-project","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"would-mega-scale-datasets-further-enhance","repo_url":"https://github.com/ziqi-zhang/3D-ResNets-PyTorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"would-mega-scale-datasets-further-enhance","repo_url":"https://github.com/mindspore-ai/models/tree/master/research/cv/resnet3d","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"open-question","task_name":"Open-Ended Question Answering"},{"task_slug":"video-classification","task_name":"Video Classification"},{"task_slug":"video-recognition","task_name":"Video Recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2004.04968","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2004.04968"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rutajoshi/cs231n-project","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/3D-ResNets-Reproduction/Reference-Project","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Oichii/resnet3D_pulse","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mindspore-ai/models/tree/master/research/cv/resnet3d","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Mind23-2/MindCode-68","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/KevinPang180/3D-ResNets-PaddlePaddle","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ziqi-zhang/3D-ResNets-PyTorch","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Shito0907/3D_resnet","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kenshohara/3D-ResNets-PyTorch","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kaleab-k/VideoAT","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":2,"unverified":20},"by_repo_kind":{"official":{"samples":12,"ran":0,"repositories":1},"listed":{"samples":10,"ran":2,"repositories":3}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"213ccc8fb87ee8d9","entry":"conv1x1x1","repo":"ziqi-zhang/3D-ResNets-PyTorch","repo_kind":"listed","path":"models/resnet.py","file_url":"https://github.com/ziqi-zhang/3D-ResNets-PyTorch/blob/HEAD/models/resnet.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"213ccc8fb87ee8d9"}},{"code_sha256_prefix":"c3b215555357b9d1","entry":"conv3x3x3","repo":"ziqi-zhang/3D-ResNets-PyTorch","repo_kind":"listed","path":"models/resnet.py","file_url":"https://github.com/ziqi-zhang/3D-ResNets-PyTorch/blob/HEAD/models/resnet.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c3b215555357b9d1"}},{"code_sha256_prefix":"f0c9a29156911331","entry":"accuracy","repo":"Oichii/resnet3D_pulse","repo_kind":"listed","path":"ResNet_train.py","file_url":"https://github.com/Oichii/resnet3D_pulse/blob/HEAD/ResNet_train.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f0c9a29156911331"}},{"code_sha256_prefix":"b471bd70ae7033b7","entry":"conv1x3x3","repo":"kenshohara/3D-ResNets-PyTorch","repo_kind":"official","path":"models/resnet2p1d.py","file_url":"https://github.com/kenshohara/3D-ResNets-PyTorch/blob/HEAD/models/resnet2p1d.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b471bd70ae7033b7"}},{"code_sha256_prefix":"69cf8d9e032b3251","entry":"conv3x1x1","repo":"kenshohara/3D-ResNets-PyTorch","repo_kind":"official","path":"models/resnet2p1d.py","file_url":"https://github.com/kenshohara/3D-ResNets-PyTorch/blob/HEAD/models/resnet2p1d.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"69cf8d9e032b3251"}},{"code_sha256_prefix":"392a1ed5b8624559","entry":"generate_model","repo":"ziqi-zhang/3D-ResNets-PyTorch","repo_kind":"listed","path":"models/resnet.py","file_url":"https://github.com/ziqi-zhang/3D-ResNets-PyTorch/blob/HEAD/models/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"392a1ed5b8624559"}},{"code_sha256_prefix":"f1faf2b3a11cd3c6","entry":"generate_model","repo":"kenshohara/3D-ResNets-PyTorch","repo_kind":"official","path":"models/densenet.py","file_url":"https://github.com/kenshohara/3D-ResNets-PyTorch/blob/HEAD/models/densenet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f1faf2b3a11cd3c6"}},{"code_sha256_prefix":"45343fb6310c6c49","entry":"generate_model","repo":"kenshohara/3D-ResNets-PyTorch","repo_kind":"official","path":"models/pre_act_resnet.py","file_url":"https://github.com/kenshohara/3D-ResNets-PyTorch/blob/HEAD/models/pre_act_resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"45343fb6310c6c49"}},{"code_sha256_prefix":"c03ab7046d986487","entry":"generate_model","repo":"Oichii/resnet3D_pulse","repo_kind":"listed","path":"ResNet_model.py","file_url":"https://github.com/Oichii/resnet3D_pulse/blob/HEAD/ResNet_model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c03ab7046d986487"}},{"code_sha256_prefix":"0b3b02ceef079aae","entry":"get_class_labels","repo":"kenshohara/3D-ResNets-PyTorch","repo_kind":"official","path":"datasets/activitynet.py","file_url":"https://github.com/kenshohara/3D-ResNets-PyTorch/blob/HEAD/datasets/activitynet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0b3b02ceef079aae"}},{"code_sha256_prefix":"c59254c1a5526d41","entry":"get_fine_tuning_parameters","repo":"kenshohara/3D-ResNets-PyTorch","repo_kind":"official","path":"model.py","file_url":"https://github.com/kenshohara/3D-ResNets-PyTorch/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c59254c1a5526d41"}},{"code_sha256_prefix":"d35f7d3b70ef9b70","entry":"get_mean_std","repo":"kenshohara/3D-ResNets-PyTorch","repo_kind":"official","path":"mean.py","file_url":"https://github.com/kenshohara/3D-ResNets-PyTorch/blob/HEAD/mean.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d35f7d3b70ef9b70"}},{"code_sha256_prefix":"c347904eafd0d22e","entry":"get_module_name","repo":"kenshohara/3D-ResNets-PyTorch","repo_kind":"official","path":"model.py","file_url":"https://github.com/kenshohara/3D-ResNets-PyTorch/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c347904eafd0d22e"}},{"code_sha256_prefix":"92f569d7dc7c5f17","entry":"get_n_frames","repo":"kenshohara/3D-ResNets-PyTorch","repo_kind":"official","path":"datasets/activitynet.py","file_url":"https://github.com/kenshohara/3D-ResNets-PyTorch/blob/HEAD/datasets/activitynet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"92f569d7dc7c5f17"}},{"code_sha256_prefix":"8dd3e55b4ebf8cc6","entry":"get_video_ids_annotations_and_fps","repo":"kenshohara/3D-ResNets-PyTorch","repo_kind":"official","path":"datasets/activitynet.py","file_url":"https://github.com/kenshohara/3D-ResNets-PyTorch/blob/HEAD/datasets/activitynet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8dd3e55b4ebf8cc6"}},{"code_sha256_prefix":"c0e7431aff3ed281","entry":"get_video_results","repo":"kenshohara/3D-ResNets-PyTorch","repo_kind":"official","path":"inference.py","file_url":"https://github.com/kenshohara/3D-ResNets-PyTorch/blob/HEAD/inference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c0e7431aff3ed281"}},{"code_sha256_prefix":"1f00a5b96d91ec0d","entry":"group_multi_scale_crop","repo":"KevinPang180/3D-ResNets-PaddlePaddle","repo_kind":"listed","path":"reader.py","file_url":"https://github.com/KevinPang180/3D-ResNets-PaddlePaddle/blob/HEAD/reader.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1f00a5b96d91ec0d"}},{"code_sha256_prefix":"865572e58d23a992","entry":"group_random_crop","repo":"KevinPang180/3D-ResNets-PaddlePaddle","repo_kind":"listed","path":"reader.py","file_url":"https://github.com/KevinPang180/3D-ResNets-PaddlePaddle/blob/HEAD/reader.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"865572e58d23a992"}},{"code_sha256_prefix":"e5bd557fa83521ca","entry":"group_random_flip","repo":"KevinPang180/3D-ResNets-PaddlePaddle","repo_kind":"listed","path":"reader.py","file_url":"https://github.com/KevinPang180/3D-ResNets-PaddlePaddle/blob/HEAD/reader.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e5bd557fa83521ca"}},{"code_sha256_prefix":"138fe18f637e2e35","entry":"image_name_formatter","repo":"kenshohara/3D-ResNets-PyTorch","repo_kind":"official","path":"dataset.py","file_url":"https://github.com/kenshohara/3D-ResNets-PyTorch/blob/HEAD/dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"138fe18f637e2e35"}},{"code_sha256_prefix":"0e03ada3bc54e4fb","entry":"merge_configs","repo":"KevinPang180/3D-ResNets-PaddlePaddle","repo_kind":"listed","path":"config.py","file_url":"https://github.com/KevinPang180/3D-ResNets-PaddlePaddle/blob/HEAD/config.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0e03ada3bc54e4fb"}},{"code_sha256_prefix":"9436376f4d71d64e","entry":"validate","repo":"Oichii/resnet3D_pulse","repo_kind":"listed","path":"ResNet_train.py","file_url":"https://github.com/Oichii/resnet3D_pulse/blob/HEAD/ResNet_train.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9436376f4d71d64e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}