{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/resgrad-residual-denoising-diffusion","title":"ResGrad: Residual Denoising Diffusion Probabilistic Models for Text to Speech","arxiv_id":"2212.14518","date":"2022-12-30","proceeding":null,"authors":["Zehua Chen","Yihan Wu","Yichong Leng","Jiawei Chen","Haohe Liu","Xu Tan","Yang Cui","Ke Wang","Lei He","Sheng Zhao","Jiang Bian","Danilo Mandic"],"abstract":"Denoising Diffusion Probabilistic Models (DDPMs) are emerging in text-to-speech (TTS) synthesis because of their strong capability of generating high-fidelity samples. However, their iterative refinement process in high-dimensional data space results in slow inference speed, which restricts their application in real-time systems. Previous works have explored speeding up by minimizing the number of inference steps but at the cost of sample quality. In this work, to improve the inference speed for DDPM-based TTS model while achieving high sample quality, we propose ResGrad, a lightweight diffusion model which learns to refine the output spectrogram of an existing TTS model (e.g., FastSpeech 2) by predicting the residual between the model output and the corresponding ground-truth speech. ResGrad has several advantages: 1) Compare with other acceleration methods for DDPM which need to synthesize speech from scratch, ResGrad reduces the complexity of task by changing the generation target from ground-truth mel-spectrogram to the residual, resulting into a more lightweight model and thus a smaller real-time factor. 2) ResGrad is employed in the inference process of the existing TTS model in a plug-and-play way, without re-training this model. We verify ResGrad on the single-speaker dataset LJSpeech and two more challenging datasets with multiple speakers (LibriTTS) and high sampling rate (VCTK). Experimental results show that in comparison with other speed-up methods of DDPMs: 1) ResGrad achieves better sample quality with the same inference speed measured by real-time factor; 2) with similar speech quality, ResGrad synthesizes speech faster than baseline methods by more than 10 times. Audio samples are available at https://resgrad1.github.io/.","url_abs":"https://arxiv.org/abs/2212.14518v1","url_pdf":"https://arxiv.org/pdf/2212.14518v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"resgrad-residual-denoising-diffusion","repo_url":"https://github.com/majidAdibian77/ResGrad","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"text-to-speech","task_name":"Text to Speech"},{"task_slug":"text-to-speech-1","task_name":"text-to-speech"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2212.14518","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2212.14518"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/majidAdibian77/ResGrad","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":1,"unverified":9},"by_repo_kind":{"listed":{"samples":10,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a26f85d7c72ef39a","entry":"get_padding","repo":"majidAdibian77/ResGrad","repo_kind":"listed","path":"vocoder/models.py","file_url":"https://github.com/majidAdibian77/ResGrad/blob/HEAD/vocoder/models.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a26f85d7c72ef39a"}},{"code_sha256_prefix":"81fb0c29833351df","entry":"create_dataset","repo":"majidAdibian77/ResGrad","repo_kind":"listed","path":"resgrad/data.py","file_url":"https://github.com/majidAdibian77/ResGrad/blob/HEAD/resgrad/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"81fb0c29833351df"}},{"code_sha256_prefix":"b79fe21c0e167974","entry":"crop_masked_values","repo":"majidAdibian77/ResGrad","repo_kind":"listed","path":"resgrad/utils.py","file_url":"https://github.com/majidAdibian77/ResGrad/blob/HEAD/resgrad/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b79fe21c0e167974"}},{"code_sha256_prefix":"f56eb8bc75bfa418","entry":"get_file_name","repo":"majidAdibian77/ResGrad","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/majidAdibian77/ResGrad/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f56eb8bc75bfa418"}},{"code_sha256_prefix":"9211d7c738c4dfe9","entry":"get_noise","repo":"majidAdibian77/ResGrad","repo_kind":"listed","path":"resgrad/model/diffusion.py","file_url":"https://github.com/majidAdibian77/ResGrad/blob/HEAD/resgrad/model/diffusion.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9211d7c738c4dfe9"}},{"code_sha256_prefix":"bfea89f6d4ff61c6","entry":"infer","repo":"majidAdibian77/ResGrad","repo_kind":"listed","path":"resgrad/inference.py","file_url":"https://github.com/majidAdibian77/ResGrad/blob/HEAD/resgrad/inference.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bfea89f6d4ff61c6"}},{"code_sha256_prefix":"8db92bb5bc44496b","entry":"load_yaml_file","repo":"majidAdibian77/ResGrad","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/majidAdibian77/ResGrad/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8db92bb5bc44496b"}},{"code_sha256_prefix":"ef79a17b3941ca58","entry":"plot_tensor","repo":"majidAdibian77/ResGrad","repo_kind":"listed","path":"resgrad/utils.py","file_url":"https://github.com/majidAdibian77/ResGrad/blob/HEAD/resgrad/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ef79a17b3941ca58"}},{"code_sha256_prefix":"6953388e80e5bd47","entry":"read_input_data","repo":"majidAdibian77/ResGrad","repo_kind":"listed","path":"resgrad_data.py","file_url":"https://github.com/majidAdibian77/ResGrad/blob/HEAD/resgrad_data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6953388e80e5bd47"}},{"code_sha256_prefix":"29d749e3e95fa41d","entry":"save_figure_to_numpy","repo":"majidAdibian77/ResGrad","repo_kind":"listed","path":"resgrad/utils.py","file_url":"https://github.com/majidAdibian77/ResGrad/blob/HEAD/resgrad/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"29d749e3e95fa41d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}