{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/safespeech-robust-and-universal-voice","title":"SafeSpeech: Robust and Universal Voice Protection Against Malicious Speech Synthesis","arxiv_id":"2504.09839","date":"2025-04-14","proceeding":null,"authors":["Zhisheng Zhang","Derui Wang","Qianyi Yang","Pengyang Huang","Junhan Pu","Yuxin Cao","Kai Ye","Jie Hao","Yixian Yang"],"abstract":"Speech synthesis technology has brought great convenience, while the widespread usage of realistic deepfake audio has triggered hazards. Malicious adversaries may unauthorizedly collect victims' speeches and clone a similar voice for illegal exploitation (\\textit{e.g.}, telecom fraud). However, the existing defense methods cannot effectively prevent deepfake exploitation and are vulnerable to robust training techniques. Therefore, a more effective and robust data protection method is urgently needed. In response, we propose a defensive framework, \\textit{\\textbf{SafeSpeech}}, which protects the users' audio before uploading by embedding imperceptible perturbations on original speeches to prevent high-quality synthetic speech. In SafeSpeech, we devise a robust and universal proactive protection technique, \\textbf{S}peech \\textbf{PE}rturbative \\textbf{C}oncealment (\\textbf{SPEC}), that leverages a surrogate model to generate universally applicable perturbation for generative synthetic models. Moreover, we optimize the human perception of embedded perturbation in terms of time and frequency domains. To evaluate our method comprehensively, we conduct extensive experiments across advanced models and datasets, both subjectively and objectively. Our experimental results demonstrate that SafeSpeech achieves state-of-the-art (SOTA) voice protection effectiveness and transferability and is highly robust against advanced adaptive adversaries. Moreover, SafeSpeech has real-time capability in real-world tests. The source code is available at \\href{https://github.com/wxzyd123/SafeSpeech}{https://github.com/wxzyd123/SafeSpeech}.","url_abs":"https://arxiv.org/abs/2504.09839v1","url_pdf":"https://arxiv.org/pdf/2504.09839v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"safespeech-robust-and-universal-voice","repo_url":"https://github.com/wxzyd123/safespeech","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"face-swapping","task_name":"Face Swapping"},{"task_slug":"speech-synthesis","task_name":"Speech Synthesis"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2504.09839","atlas_url":"https://app.syntology.ai/?focus=2504.09839","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2504.09839"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wxzyd123/safespeech","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"unverified":3},"by_repo_kind":{"official":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4a5459614b02ff4c","entry":"copyStateDict","repo":"wxzyd123/safespeech","repo_kind":"official","path":"bert_vits2/compress_model.py","file_url":"https://github.com/wxzyd123/safespeech/blob/HEAD/bert_vits2/compress_model.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4a5459614b02ff4c"}},{"code_sha256_prefix":"896f7aae82a62114","entry":"discriminator_loss","repo":"wxzyd123/safespeech","repo_kind":"official","path":"bert_vits2/losses.py","file_url":"https://github.com/wxzyd123/safespeech/blob/HEAD/bert_vits2/losses.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"896f7aae82a62114"}},{"code_sha256_prefix":"24e8b4731dc3e070","entry":"feature_loss","repo":"wxzyd123/safespeech","repo_kind":"official","path":"bert_vits2/losses.py","file_url":"https://github.com/wxzyd123/safespeech/blob/HEAD/bert_vits2/losses.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"24e8b4731dc3e070"}},{"code_sha256_prefix":"6191fd9efbdd55d4","entry":"generator_loss","repo":"wxzyd123/safespeech","repo_kind":"official","path":"bert_vits2/losses.py","file_url":"https://github.com/wxzyd123/safespeech/blob/HEAD/bert_vits2/losses.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6191fd9efbdd55d4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}