{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/improving-fairness-for-spoken-language","title":"Improving fairness for spoken language understanding in atypical speech with Text-to-Speech","arxiv_id":"2311.10149","date":"2023-11-16","proceeding":null,"authors":["Helin Wang","Venkatesh Ravichandran","Milind Rao","Becky Lammers","Myra Sydnor","Nicholas Maragakis","Ankur A. Butala","Jayne Zhang","Lora Clawson","Victoria Chovaz","Laureano Moro-Velazquez"],"abstract":"Spoken language understanding (SLU) systems often exhibit suboptimal performance in processing atypical speech, typically caused by neurological conditions and motor impairments. Recent advancements in Text-to-Speech (TTS) synthesis-based augmentation for more fair SLU have struggled to accurately capture the unique vocal characteristics of atypical speakers, largely due to insufficient data. To address this issue, we present a novel data augmentation method for atypical speakers by finetuning a TTS model, called Aty-TTS. Aty-TTS models speaker and atypical characteristics via knowledge transferring from a voice conversion model. Then, we use the augmented data to train SLU models adapted to atypical speech. To train these data augmentation models and evaluate the resulting SLU systems, we have collected a new atypical speech dataset containing intent annotation. Both objective and subjective assessments validate that Aty-TTS is capable of generating high-quality atypical speech. Furthermore, it serves as an effective data augmentation strategy, contributing to more fair SLU systems that can better accommodate individuals with atypical speech patterns.","url_abs":"https://arxiv.org/abs/2311.10149v1","url_pdf":"https://arxiv.org/pdf/2311.10149v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"improving-fairness-for-spoken-language","repo_url":"https://github.com/wanghelin1997/aty-tts","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"fairness","task_name":"Fairness"},{"task_slug":"spoken-language-understanding","task_name":"Spoken Language Understanding"},{"task_slug":"text-to-speech","task_name":"Text to Speech"},{"task_slug":"voice-conversion","task_name":"Voice Conversion"},{"task_slug":"text-to-speech-1","task_name":"text-to-speech"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}