{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/whispering-under-the-eaves-protecting-user","title":"Whispering Under the Eaves: Protecting User Privacy Against Commercial and LLM-powered Automatic Speech Recognition Systems","arxiv_id":"2504.00858","date":"2025-04-01","proceeding":null,"authors":["Weifei Jin","Yuxin Cao","Junjie Su","Derui Wang","Yedi Zhang","Minhui Xue","Jie Hao","Jin Song Dong","Yixian Yang"],"abstract":"The widespread application of automatic speech recognition (ASR) supports large-scale voice surveillance, raising concerns about privacy among users. In this paper, we concentrate on using adversarial examples to mitigate unauthorized disclosure of speech privacy thwarted by potential eavesdroppers in speech communications. While audio adversarial examples have demonstrated the capability to mislead ASR models or evade ASR surveillance, they are typically constructed through time-intensive offline optimization, restricting their practicality in real-time voice communication. Recent work overcame this limitation by generating universal adversarial perturbations (UAPs) and enhancing their transferability for black-box scenarios. However, they introduced excessive noise that significantly degrades audio quality and affects human perception, thereby limiting their effectiveness in practical scenarios. To address this limitation and protect live users' speech against ASR systems, we propose a novel framework, AudioShield. Central to this framework is the concept of Transferable Universal Adversarial Perturbations in the Latent Space (LS-TUAP). By transferring the perturbations to the latent space, the audio quality is preserved to a large extent. Additionally, we propose target feature adaptation to enhance the transferability of UAPs by embedding target text features into the perturbations. Comprehensive evaluation on four commercial ASR APIs (Google, Amazon, iFlytek, and Alibaba), three voice assistants, two LLM-powered ASR and one NN-based ASR demonstrates the protection superiority of AudioShield over existing competitors, and both objective and subjective evaluations indicate that AudioShield significantly improves the audio quality. Moreover, AudioShield also shows high effectiveness in real-time end-to-end scenarios, and demonstrates strong resilience against adaptive countermeasures.","url_abs":"https://arxiv.org/abs/2504.00858v1","url_pdf":"https://arxiv.org/pdf/2504.00858v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"whispering-under-the-eaves-protecting-user","repo_url":"https://github.com/WeifeiJin/AudioShield","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"automatic-speech-recognition-2","task_name":"Automatic Speech Recognition"},{"task_slug":"automatic-speech-recognition","task_name":"Automatic Speech Recognition (ASR)"},{"task_slug":"speech-recognition","task_name":"Speech Recognition"},{"task_slug":"speech-recognition-1","task_name":"speech-recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2504.00858","atlas_url":"https://app.syntology.ai/?focus=2504.00858","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2504.00858"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/WeifeiJin/AudioShield","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":2,"ran":3,"unverified":6},"by_repo_kind":{"official":{"samples":11,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"59ae0708b1e8d94b","entry":"convert_pad_shape","repo":"WeifeiJin/AudioShield","repo_kind":"official","path":"commons.py","file_url":"https://github.com/WeifeiJin/AudioShield/blob/HEAD/commons.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"59ae0708b1e8d94b"}},{"code_sha256_prefix":"a26f85d7c72ef39a","entry":"get_padding","repo":"WeifeiJin/AudioShield","repo_kind":"official","path":"commons.py","file_url":"https://github.com/WeifeiJin/AudioShield/blob/HEAD/commons.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a26f85d7c72ef39a"}},{"code_sha256_prefix":"321604dda57bb02d","entry":"latest_checkpoint_path","repo":"WeifeiJin/AudioShield","repo_kind":"official","path":"utils.py","file_url":"https://github.com/WeifeiJin/AudioShield/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"321604dda57bb02d"}},{"code_sha256_prefix":"c647283e53cacacf","entry":"load_checkpoint","repo":"WeifeiJin/AudioShield","repo_kind":"official","path":"utils.py","file_url":"https://github.com/WeifeiJin/AudioShield/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c647283e53cacacf"}},{"code_sha256_prefix":"70a7614b3b3eb41f","entry":"unconstrained_rational_quadratic_spline","repo":"WeifeiJin/AudioShield","repo_kind":"official","path":"transforms.py","file_url":"https://github.com/WeifeiJin/AudioShield/blob/HEAD/transforms.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"70a7614b3b3eb41f"}},{"code_sha256_prefix":"262807b9eb9706a5","entry":"dynamic_range_compression_torch","repo":"WeifeiJin/AudioShield","repo_kind":"official","path":"mel_processing.py","file_url":"https://github.com/WeifeiJin/AudioShield/blob/HEAD/mel_processing.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"262807b9eb9706a5"}},{"code_sha256_prefix":"78c12e563d5af53d","entry":"dynamic_range_decompression_torch","repo":"WeifeiJin/AudioShield","repo_kind":"official","path":"mel_processing.py","file_url":"https://github.com/WeifeiJin/AudioShield/blob/HEAD/mel_processing.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"78c12e563d5af53d"}},{"code_sha256_prefix":"c01a3efb274ff778","entry":"intersperse","repo":"WeifeiJin/AudioShield","repo_kind":"official","path":"commons.py","file_url":"https://github.com/WeifeiJin/AudioShield/blob/HEAD/commons.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c01a3efb274ff778"}},{"code_sha256_prefix":"ba6cafc4d658c14f","entry":"piecewise_rational_quadratic_transform","repo":"WeifeiJin/AudioShield","repo_kind":"official","path":"transforms.py","file_url":"https://github.com/WeifeiJin/AudioShield/blob/HEAD/transforms.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ba6cafc4d658c14f"}},{"code_sha256_prefix":"e85d747983c4f5d8","entry":"searchsorted","repo":"WeifeiJin/AudioShield","repo_kind":"official","path":"transforms.py","file_url":"https://github.com/WeifeiJin/AudioShield/blob/HEAD/transforms.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e85d747983c4f5d8"}},{"code_sha256_prefix":"26b87488d429896a","entry":"spectral_normalize_torch","repo":"WeifeiJin/AudioShield","repo_kind":"official","path":"mel_processing.py","file_url":"https://github.com/WeifeiJin/AudioShield/blob/HEAD/mel_processing.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"26b87488d429896a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}