{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/wavlm-model-ensemble-for-audio-deepfake","title":"WavLM model ensemble for audio deepfake detection","arxiv_id":"2408.07414","date":"2024-08-14","proceeding":null,"authors":["David Combei","Adriana Stan","Dan Oneata","Horia Cucu"],"abstract":"Audio deepfake detection has become a pivotal task over the last couple of years, as many recent speech synthesis and voice cloning systems generate highly realistic speech samples, thus enabling their use in malicious activities. In this paper we address the issue of audio deepfake detection as it was set in the ASVspoof5 challenge. First, we benchmark ten types of pretrained representations and show that the self-supervised representations stemming from the wav2vec2 and wavLM families perform best. Of the two, wavLM is better when restricting the pretraining data to LibriSpeech, as required by the challenge rules. To further improve performance, we finetune the wavLM model for the deepfake detection task. We extend the ASVspoof5 dataset with samples from other deepfake detection datasets and apply data augmentation. Our final challenge submission consists of a late fusion combination of four models and achieves an equal error rate of 6.56% and 17.08% on the two evaluation sets.","url_abs":"https://arxiv.org/abs/2408.07414v1","url_pdf":"https://arxiv.org/pdf/2408.07414v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"wavlm-model-ensemble-for-audio-deepfake","repo_url":"https://github.com/MindCode-4/code-5/tree/main/wavlm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"wavlm-model-ensemble-for-audio-deepfake","repo_url":"https://github.com/pwc-1/Paper-9/tree/main/5/wavlm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"audio-deepfake-detection","task_name":"Audio Deepfake Detection"},{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"deepfake-detection","task_name":"DeepFake Detection"},{"task_slug":"face-swapping","task_name":"Face Swapping"},{"task_slug":"speech-synthesis","task_name":"Speech Synthesis"},{"task_slug":"voice-cloning","task_name":"Voice Cloning"},{"task_slug":"model","task_name":"model"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}