{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/combining-efficientnet-and-vision","title":"Combining EfficientNet and Vision Transformers for Video Deepfake Detection","arxiv_id":"2107.02612","date":"2021-07-06","proceeding":null,"authors":["Davide Coccomini","Nicola Messina","Claudio Gennaro","Fabrizio Falchi"],"abstract":"Deepfakes are the result of digital manipulation to forge realistic yet fake imagery. With the astonishing advances in deep generative models, fake images or videos are nowadays obtained using variational autoencoders (VAEs) or Generative Adversarial Networks (GANs). These technologies are becoming more accessible and accurate, resulting in fake videos that are very difficult to be detected. Traditionally, Convolutional Neural Networks (CNNs) have been used to perform video deepfake detection, with the best results obtained using methods based on EfficientNet B7. In this study, we focus on video deep fake detection on faces, given that most methods are becoming extremely accurate in the generation of realistic human faces. Specifically, we combine various types of Vision Transformers with a convolutional EfficientNet B0 used as a feature extractor, obtaining comparable results with some very recent methods that use Vision Transformers. Differently from the state-of-the-art approaches, we use neither distillation nor ensemble methods. Furthermore, we present a straightforward inference procedure based on a simple voting scheme for handling multiple faces in the same video shot. The best model achieved an AUC of 0.951 and an F1 score of 88.0%, very close to the state-of-the-art on the DeepFake Detection Challenge (DFDC).","url_abs":"https://arxiv.org/abs/2107.02612v2","url_pdf":"https://arxiv.org/pdf/2107.02612v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"combining-efficientnet-and-vision","repo_url":"https://github.com/davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"combining-efficientnet-and-vision","repo_url":"https://github.com/davide-coccomini/Combining-EfficientNet-and-Vision-Transformersfor-Video-Deepfake-Detection","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"combining-efficientnet-and-vision","repo_url":"https://github.com/davide-coccomini/mintime-multi-identity-size-invariant-timesformer-for-video-deepfake-detection","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"deepfake-detection","task_name":"DeepFake Detection"},{"task_slug":"face-swapping","task_name":"Face Swapping"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"depthwise-convolution","method_name":"Depthwise Convolution"},{"method_slug":"depthwise-separable-convolution","method_name":"Depthwise Separable Convolution"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"inverted-residual-block","method_name":"Inverted Residual Block"},{"method_slug":"pointwise-convolution","method_name":"Pointwise Convolution"},{"method_slug":"rmsprop","method_name":"RMSProp"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"squeeze-and-excitation-block","method_name":"Squeeze-and-Excitation Block"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/deepfake-detection-on-dfdc","task":"DeepFake Detection","dataset":"DFDC","model":"Cross Efficient Vision Transformer","rank_in_archive_order":1,"of":3,"metrics":{"AUC":"0.951"},"uses_additional_data":true},{"leaderboard":"/sota/deepfake-detection-on-dfdc","task":"DeepFake Detection","dataset":"DFDC","model":"Efficient Vision Transformer","rank_in_archive_order":2,"of":3,"metrics":{"AUC":"0.919"},"uses_additional_data":true}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2107.02612","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2107.02612"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/davide-coccomini/Combining-EfficientNet-and-Vision-Transformersfor-Video-Deepfake-Detection","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/davide-coccomini/mintime-multi-identity-size-invariant-timesformer-for-video-deepfake-detection","reach":{"status":"ok"}}],"summary":{"ran_violates":2,"ran_fixture":1,"ran_honours":1,"ran":1,"ran_draft_wrong":2,"unverified":7},"by_repo_kind":{"official":{"samples":14,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"60fff7c3c400d7ff","entry":"default","repo":"davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection","repo_kind":"official","path":"cross-efficient-vit/cross_efficient_vit.py","file_url":"https://github.com/davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection/blob/HEAD/cross-efficient-vit/cross_efficient_vit.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"60fff7c3c400d7ff"}},{"code_sha256_prefix":"d3319e3d34ca90ca","entry":"drop_connect","repo":"davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection","repo_kind":"official","path":"cross-efficient-vit/efficient_net/efficientnet_pytorch/utils.py","file_url":"https://github.com/davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection/blob/HEAD/cross-efficient-vit/efficient_net/efficientnet_pytorch/utils.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d3319e3d34ca90ca"}},{"code_sha256_prefix":"9e354d875b0e94e6","entry":"efficientnet_params","repo":"davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection","repo_kind":"official","path":"cross-efficient-vit/efficient_net/tf_to_pytorch/convert_tf_to_pt/original_tf/efficientnet_builder.py","file_url":"https://github.com/davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection/blob/HEAD/cross-efficient-vit/efficient_net/tf_to_pytorch/convert_tf_to_pt/original_tf/efficientnet_builder.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9e354d875b0e94e6"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection","repo_kind":"official","path":"cross-efficient-vit/cross_efficient_vit.py","file_url":"https://github.com/davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection/blob/HEAD/cross-efficient-vit/cross_efficient_vit.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"3fd63f015c463947","entry":"get_img_id","repo":"davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection","repo_kind":"official","path":"cross-efficient-vit/efficient_net/sotabench.py","file_url":"https://github.com/davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection/blob/HEAD/cross-efficient-vit/efficient_net/sotabench.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3fd63f015c463947"}},{"code_sha256_prefix":"f15a49337e69e937","entry":"round_filters","repo":"davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection","repo_kind":"official","path":"cross-efficient-vit/efficient_net/efficientnet_pytorch/utils.py","file_url":"https://github.com/davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection/blob/HEAD/cross-efficient-vit/efficient_net/efficientnet_pytorch/utils.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f15a49337e69e937"}},{"code_sha256_prefix":"dbc0ca08d119a5a0","entry":"round_repeats","repo":"davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection","repo_kind":"official","path":"cross-efficient-vit/efficient_net/efficientnet_pytorch/utils.py","file_url":"https://github.com/davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection/blob/HEAD/cross-efficient-vit/efficient_net/efficientnet_pytorch/utils.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dbc0ca08d119a5a0"}},{"code_sha256_prefix":"a039ef466f295d02","entry":"conv_kernel_initializer","repo":"davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection","repo_kind":"official","path":"cross-efficient-vit/efficient_net/tf_to_pytorch/convert_tf_to_pt/original_tf/efficientnet_model.py","file_url":"https://github.com/davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection/blob/HEAD/cross-efficient-vit/efficient_net/tf_to_pytorch/convert_tf_to_pt/original_tf/efficientnet_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a039ef466f295d02"}},{"code_sha256_prefix":"31b6867bec9093e3","entry":"custom_round","repo":"davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection","repo_kind":"official","path":"cross-efficient-vit/utils.py","file_url":"https://github.com/davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection/blob/HEAD/cross-efficient-vit/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"31b6867bec9093e3"}},{"code_sha256_prefix":"5713b9612b70a728","entry":"custom_round","repo":"davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection","repo_kind":"official","path":"efficient-vit/utils.py","file_url":"https://github.com/davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection/blob/HEAD/efficient-vit/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5713b9612b70a728"}},{"code_sha256_prefix":"ec6986123a6a4894","entry":"dense_kernel_initializer","repo":"davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection","repo_kind":"official","path":"cross-efficient-vit/efficient_net/tf_to_pytorch/convert_tf_to_pt/original_tf/efficientnet_model.py","file_url":"https://github.com/davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection/blob/HEAD/cross-efficient-vit/efficient_net/tf_to_pytorch/convert_tf_to_pt/original_tf/efficientnet_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ec6986123a6a4894"}},{"code_sha256_prefix":"441055127d2e4fa1","entry":"resize","repo":"davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection","repo_kind":"official","path":"cross-efficient-vit/utils.py","file_url":"https://github.com/davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection/blob/HEAD/cross-efficient-vit/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"441055127d2e4fa1"}},{"code_sha256_prefix":"abe000c0df58d25c","entry":"superpixel_kernel_initializer","repo":"davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection","repo_kind":"official","path":"cross-efficient-vit/efficient_net/tf_to_pytorch/convert_tf_to_pt/original_tf/efficientnet_model.py","file_url":"https://github.com/davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection/blob/HEAD/cross-efficient-vit/efficient_net/tf_to_pytorch/convert_tf_to_pt/original_tf/efficientnet_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"abe000c0df58d25c"}},{"code_sha256_prefix":"918db160732a16d0","entry":"swish","repo":"davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection","repo_kind":"official","path":"cross-efficient-vit/efficient_net/tf_to_pytorch/convert_tf_to_pt/original_tf/efficientnet_builder.py","file_url":"https://github.com/davide-coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection/blob/HEAD/cross-efficient-vit/efficient_net/tf_to_pytorch/convert_tf_to_pt/original_tf/efficientnet_builder.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"918db160732a16d0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}