{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/efficient-neural-audio-synthesis","title":"Efficient Neural Audio Synthesis","arxiv_id":"1802.08435","date":"2018-02-23","proceeding":"ICML 2018 7","authors":["Nal Kalchbrenner","Erich Elsen","Karen Simonyan","Seb Noury","Norman Casagrande","Edward Lockhart","Florian Stimberg","Aaron van den Oord","Sander Dieleman","Koray Kavukcuoglu"],"abstract":"Sequential models achieve state-of-the-art results in audio, visual and\ntextual domains with respect to both estimating the data distribution and\ngenerating high-quality samples. Efficient sampling for this class of models\nhas however remained an elusive problem. With a focus on text-to-speech\nsynthesis, we describe a set of general techniques for reducing sampling time\nwhile maintaining high output quality. We first describe a single-layer\nrecurrent neural network, the WaveRNN, with a dual softmax layer that matches\nthe quality of the state-of-the-art WaveNet model. The compact form of the\nnetwork makes it possible to generate 24kHz 16-bit audio 4x faster than real\ntime on a GPU. Second, we apply a weight pruning technique to reduce the number\nof weights in the WaveRNN. We find that, for a constant number of parameters,\nlarge sparse networks perform better than small dense networks and this\nrelationship holds for sparsity levels beyond 96%. The small number of weights\nin a Sparse WaveRNN makes it possible to sample high-fidelity audio on a mobile\nCPU in real time. Finally, we propose a new generation scheme based on\nsubscaling that folds a long sequence into a batch of shorter sequences and\nallows one to generate multiple samples at once. The Subscale WaveRNN produces\n16 samples per step without loss of quality and offers an orthogonal method for\nincreasing sampling efficiency.","url_abs":"http://arxiv.org/abs/1802.08435v2","url_pdf":"http://arxiv.org/pdf/1802.08435v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficient-neural-audio-synthesis","repo_url":"https://github.com/CorentinJ/Real-Time-Voice-Cloning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"efficient-neural-audio-synthesis","repo_url":"https://github.com/GPUPhobia/vocal-mask","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"efficient-neural-audio-synthesis","repo_url":"https://github.com/OlaWod/my-wavernn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"efficient-neural-audio-synthesis","repo_url":"https://github.com/Rongjiehuang/Multiband-WaveRNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"efficient-neural-audio-synthesis","repo_url":"https://github.com/anandaswarup/waveRNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"efficient-neural-audio-synthesis","repo_url":"https://github.com/caizexin/tf_multispeakerTTS_fc","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"efficient-neural-audio-synthesis","repo_url":"https://github.com/ciaua/score_lyrics_free_svg","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"efficient-neural-audio-synthesis","repo_url":"https://github.com/dipjyoti92/SC-WaveRNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"efficient-neural-audio-synthesis","repo_url":"https://github.com/dipjyoti92/TTS-Style-Transfer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"efficient-neural-audio-synthesis","repo_url":"https://github.com/fatchord/WaveRNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"efficient-neural-audio-synthesis","repo_url":"https://github.com/google/lyra","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"efficient-neural-audio-synthesis","repo_url":"https://github.com/mkotha/WaveRNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"efficient-neural-audio-synthesis","repo_url":"https://github.com/tiberiu44/TTS-Cube","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"efficient-neural-audio-synthesis","repo_url":"https://github.com/tigthor/Voice-Cloning-AI","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"efficient-neural-audio-synthesis","repo_url":"https://github.com/PaddlePaddle/PaddleSpeech","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"paddle","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"efficient-neural-audio-synthesis","repo_url":"https://github.com/coqui-ai/TTS","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MPL-2.0"}}],"tasks":[{"task_slug":"audio-synthesis","task_name":"Audio Synthesis"},{"task_slug":null,"task_name":"CPU"},{"task_slug":null,"task_name":"GPU"},{"task_slug":"speech-synthesis","task_name":"Speech Synthesis"},{"task_slug":"text-to-speech","task_name":"Text to Speech"},{"task_slug":"text-to-speech-synthesis","task_name":"Text-To-Speech Synthesis"},{"task_slug":"text-to-speech-1","task_name":"text-to-speech"}],"methods":[{"method_slug":"pruning","method_name":"Pruning"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"},{"method_slug":"wavernn","method_name":"WaveRNN"}],"datasets_introduced":[],"methods_introduced":[{"slug":"wavernn","name":"WaveRNN","full_name":"WaveRNN"}],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1802.08435","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1802.08435"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google/lyra","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Rongjiehuang/Multiband-WaveRNN","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dipjyoti92/SC-WaveRNN","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/PaddlePaddle/PaddleSpeech","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tiberiu44/TTS-Cube","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/caizexin/tf_multispeakerTTS_fc","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/anandaswarup/waveRNN","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/OlaWod/my-wavernn","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mkotha/WaveRNN","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/CorentinJ/Real-Time-Voice-Cloning","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tigthor/Voice-Cloning-AI","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ciaua/score_lyrics_free_svg","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/coqui-ai/TTS","reach":{"status":"ok","spdx":"MPL-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dipjyoti92/TTS-Style-Transfer","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/fatchord/WaveRNN","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/GPUPhobia/vocal-mask","reach":null}],"summary":{"ran_honours":2,"ran_draft_wrong":1},"by_repo_kind":{"listed":{"samples":2,"ran":2,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"f0b0dd80e32ae81d","entry":"cosine_decay","repo":"Rongjiehuang/Multiband-WaveRNN","repo_kind":"listed","path":"train_wavernn.py","file_url":"https://github.com/Rongjiehuang/Multiband-WaveRNN/blob/HEAD/train_wavernn.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f0b0dd80e32ae81d"}},{"code_sha256_prefix":"e7889f3a0209e24c","entry":"filter_none","repo":"mkotha/WaveRNN","repo_kind":"listed","path":"layers/overtone.py","file_url":"https://github.com/mkotha/WaveRNN/blob/HEAD/layers/overtone.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e7889f3a0209e24c"}},{"code_sha256_prefix":"2af6a131384db5c8","entry":"np_now","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"2af6a131384db5c8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}