{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/moe-ffd-mixture-of-experts-for-generalized","title":"MoE-FFD: Mixture of Experts for Generalized and Parameter-Efficient Face Forgery Detection","arxiv_id":"2404.08452","date":"2024-04-12","proceeding":null,"authors":["Chenqi Kong","Anwei Luo","Peijun Bao","Yi Yu","Haoliang Li","Zengwei Zheng","Shiqi Wang","Alex C. Kot"],"abstract":"Deepfakes have recently raised significant trust issues and security concerns among the public. Compared to CNN face forgery detectors, ViT-based methods take advantage of the expressivity of transformers, achieving superior detection performance. However, these approaches still exhibit the following limitations: (1) Fully fine-tuning ViT-based models from ImageNet weights demands substantial computational and storage resources; (2) ViT-based methods struggle to capture local forgery clues, leading to model bias; (3) These methods limit their scope on only one or few face forgery features, resulting in limited generalizability. To tackle these challenges, this work introduces Mixture-of-Experts modules for Face Forgery Detection (MoE-FFD), a generalized yet parameter-efficient ViT-based approach. MoE-FFD only updates lightweight Low-Rank Adaptation (LoRA) and Adapter layers while keeping the ViT backbone frozen, thereby achieving parameter-efficient training. Moreover, MoE-FFD leverages the expressivity of transformers and local priors of CNNs to simultaneously extract global and local forgery clues. Additionally, novel MoE modules are designed to scale the model's capacity and smartly select optimal forgery experts, further enhancing forgery detection performance. Our proposed learning scheme can be seamlessly adapted to various transformer backbones in a plug-and-play manner. Extensive experimental results demonstrate that the proposed method achieves state-of-the-art face forgery detection performance with significantly reduced parameter overhead. The code is released at: https://github.com/LoveSiameseCat/MoE-FFD.","url_abs":"https://arxiv.org/abs/2404.08452v2","url_pdf":"https://arxiv.org/pdf/2404.08452v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"moe-ffd-mixture-of-experts-for-generalized","repo_url":"https://github.com/lovesiamesecat/moe-ffd","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"mixture-of-experts","task_name":"Mixture-of-Experts"}],"methods":[{"method_slug":"adapter","method_name":"Adapter"},{"method_slug":"moe","method_name":"MoE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2404.08452","atlas_url":"https://app.syntology.ai/?focus=2404.08452","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.08452"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lovesiamesecat/moe-ffd","reach":{"status":"ok"}}],"summary":{"ran":4,"ran_fixture":1,"unverified":1},"by_repo_kind":{"official":{"samples":6,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"40767126cf816b4d","entry":"cal_metrics","repo":"lovesiamesecat/moe-ffd","repo_kind":"official","path":"utils.py","file_url":"https://github.com/lovesiamesecat/moe-ffd/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"40767126cf816b4d"}},{"code_sha256_prefix":"37be0feb3d8c8de9","entry":"cont_grad","repo":"lovesiamesecat/moe-ffd","repo_kind":"official","path":"utils.py","file_url":"https://github.com/lovesiamesecat/moe-ffd/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"37be0feb3d8c8de9"}},{"code_sha256_prefix":"7e9c935a3868a277","entry":"createConvFunc","repo":"lovesiamesecat/moe-ffd","repo_kind":"official","path":"ViT_MoE.py","file_url":"https://github.com/lovesiamesecat/moe-ffd/blob/HEAD/ViT_MoE.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7e9c935a3868a277"}},{"code_sha256_prefix":"d2ddbdaa544b49d4","entry":"find_best_threshold","repo":"lovesiamesecat/moe-ffd","repo_kind":"official","path":"utils.py","file_url":"https://github.com/lovesiamesecat/moe-ffd/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d2ddbdaa544b49d4"}},{"code_sha256_prefix":"191b2735955d18f4","entry":"resize_pos_embed","repo":"lovesiamesecat/moe-ffd","repo_kind":"official","path":"ViT_MoE.py","file_url":"https://github.com/lovesiamesecat/moe-ffd/blob/HEAD/ViT_MoE.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"191b2735955d18f4"}},{"code_sha256_prefix":"6d8a1f8a0697180f","entry":"checkpoint_filter_fn","repo":"lovesiamesecat/moe-ffd","repo_kind":"official","path":"ViT_MoE.py","file_url":"https://github.com/lovesiamesecat/moe-ffd/blob/HEAD/ViT_MoE.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6d8a1f8a0697180f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}