{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/adaspeech-adaptive-text-to-speech-for-custom-1","title":"AdaSpeech: Adaptive Text to Speech for Custom Voice","arxiv_id":"2103.00993","date":"2021-03-01","proceeding":"ICLR 2021 1","authors":["Mingjian Chen","Xu Tan","Bohan Li","Yanqing Liu","Tao Qin","Sheng Zhao","Tie-Yan Liu"],"abstract":"Custom voice, a specific text to speech (TTS) service in commercial speech platforms, aims to adapt a source TTS model to synthesize personal voice for a target speaker using few speech data. Custom voice presents two unique challenges for TTS adaptation: 1) to support diverse customers, the adaptation model needs to handle diverse acoustic conditions that could be very different from source speech data, and 2) to support a large number of customers, the adaptation parameters need to be small enough for each target speaker to reduce memory usage while maintaining high voice quality. In this work, we propose AdaSpeech, an adaptive TTS system for high-quality and efficient customization of new voices. We design several techniques in AdaSpeech to address the two challenges in custom voice: 1) To handle different acoustic conditions, we use two acoustic encoders to extract an utterance-level vector and a sequence of phoneme-level vectors from the target speech during training; in inference, we extract the utterance-level vector from a reference speech and use an acoustic predictor to predict the phoneme-level vectors. 2) To better trade off the adaptation parameters and voice quality, we introduce conditional layer normalization in the mel-spectrogram decoder of AdaSpeech, and fine-tune this part in addition to speaker embedding for adaptation. We pre-train the source TTS model on LibriTTS datasets and fine-tune it on VCTK and LJSpeech datasets (with different acoustic conditions from LibriTTS) with few adaptation data, e.g., 20 sentences, about 1 minute speech. Experiment results show that AdaSpeech achieves much better adaptation quality than baseline methods, with only about 5K specific parameters for each speaker, which demonstrates its effectiveness for custom voice. Audio samples are available at https://speechresearch.github.io/adaspeech/.","url_abs":"https://arxiv.org/abs/2103.00993v1","url_pdf":"https://arxiv.org/pdf/2103.00993v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"adaspeech-adaptive-text-to-speech-for-custom-1","repo_url":"https://github.com/rishikksh20/AdaSpeech","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"adaspeech-adaptive-text-to-speech-for-custom-1","repo_url":"https://github.com/tuanh123789/AdaSpeech","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"text-to-speech","task_name":"Text to Speech"},{"task_slug":"text-to-speech-1","task_name":"text-to-speech"}],"methods":[{"method_slug":null,"method_name":null},{"method_slug":"layer-normalization","method_name":"Layer Normalization"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2103.00993","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2103.00993"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rishikksh20/AdaSpeech","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tuanh123789/AdaSpeech","reach":null}],"summary":{"ran":9,"unverified":5},"by_repo_kind":{"listed":{"samples":14,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":14,"samples":[{"code_sha256_prefix":"5e55396cb44d6225","entry":"Condional_LayerNorm","repo":"tuanh123789/AdaSpeech","repo_kind":"listed","path":"model/adaspeech.py","file_url":"https://github.com/tuanh123789/AdaSpeech/blob/HEAD/model/adaspeech.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5e55396cb44d6225"}},{"code_sha256_prefix":"439b976b7dbb7a04","entry":"ConvNorm","repo":"tuanh123789/AdaSpeech","repo_kind":"listed","path":"model/adaspeech.py","file_url":"https://github.com/tuanh123789/AdaSpeech/blob/HEAD/model/adaspeech.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"439b976b7dbb7a04"}},{"code_sha256_prefix":"f4f002f47155dd53","entry":"LayerNorm","repo":"tuanh123789/AdaSpeech","repo_kind":"listed","path":"model/adaspeech.py","file_url":"https://github.com/tuanh123789/AdaSpeech/blob/HEAD/model/adaspeech.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f4f002f47155dd53"}},{"code_sha256_prefix":"da6a31011b387cf7","entry":"LengthRegulator","repo":"tuanh123789/AdaSpeech","repo_kind":"listed","path":"model/adaspeech.py","file_url":"https://github.com/tuanh123789/AdaSpeech/blob/HEAD/model/adaspeech.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"da6a31011b387cf7"}},{"code_sha256_prefix":"5789ba05f3bd23f9","entry":"PhonemeLevelEncoder","repo":"tuanh123789/AdaSpeech","repo_kind":"listed","path":"model/adaspeech.py","file_url":"https://github.com/tuanh123789/AdaSpeech/blob/HEAD/model/adaspeech.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5789ba05f3bd23f9"}},{"code_sha256_prefix":"ab684eb40b90a7c4","entry":"PhonemeLevelPredictor","repo":"tuanh123789/AdaSpeech","repo_kind":"listed","path":"model/adaspeech.py","file_url":"https://github.com/tuanh123789/AdaSpeech/blob/HEAD/model/adaspeech.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ab684eb40b90a7c4"}},{"code_sha256_prefix":"6c502efc22a36939","entry":"PostNet","repo":"tuanh123789/AdaSpeech","repo_kind":"listed","path":"model/adaspeech.py","file_url":"https://github.com/tuanh123789/AdaSpeech/blob/HEAD/model/adaspeech.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6c502efc22a36939"}},{"code_sha256_prefix":"45b9cbeed0ea09ae","entry":"UtteranceEncoder","repo":"tuanh123789/AdaSpeech","repo_kind":"listed","path":"model/adaspeech.py","file_url":"https://github.com/tuanh123789/AdaSpeech/blob/HEAD/model/adaspeech.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"45b9cbeed0ea09ae"}},{"code_sha256_prefix":"410ea6272ddce8dc","entry":"VariancePredictor","repo":"tuanh123789/AdaSpeech","repo_kind":"listed","path":"model/adaspeech.py","file_url":"https://github.com/tuanh123789/AdaSpeech/blob/HEAD/model/adaspeech.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"410ea6272ddce8dc"}},{"code_sha256_prefix":"866ff00f2bffde7e","entry":"AdaSpeech","repo":"tuanh123789/AdaSpeech","repo_kind":"listed","path":"model/adaspeech.py","file_url":"https://github.com/tuanh123789/AdaSpeech/blob/HEAD/model/adaspeech.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"866ff00f2bffde7e"}},{"code_sha256_prefix":"25785d34d9b88214","entry":"Decoder","repo":"tuanh123789/AdaSpeech","repo_kind":"listed","path":"model/adaspeech.py","file_url":"https://github.com/tuanh123789/AdaSpeech/blob/HEAD/model/adaspeech.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"25785d34d9b88214"}},{"code_sha256_prefix":"349c9f41be2cee91","entry":"Encoder","repo":"tuanh123789/AdaSpeech","repo_kind":"listed","path":"model/adaspeech.py","file_url":"https://github.com/tuanh123789/AdaSpeech/blob/HEAD/model/adaspeech.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"349c9f41be2cee91"}},{"code_sha256_prefix":"39b1f8c4e9f59cd3","entry":"FFTBlock","repo":"tuanh123789/AdaSpeech","repo_kind":"listed","path":"model/adaspeech.py","file_url":"https://github.com/tuanh123789/AdaSpeech/blob/HEAD/model/adaspeech.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"39b1f8c4e9f59cd3"}},{"code_sha256_prefix":"fa5a48fe6f59279c","entry":"VarianceAdaptor","repo":"tuanh123789/AdaSpeech","repo_kind":"listed","path":"model/adaspeech.py","file_url":"https://github.com/tuanh123789/AdaSpeech/blob/HEAD/model/adaspeech.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fa5a48fe6f59279c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}