Methods › Sequential › Recurrent Neural Networks › WaveRNN
WaveRNN
Introduced by Nal Kalchbrenner et al. in Efficient Neural Audio Synthesis
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
WaveRNN is a single-layer recurrent neural network for audio generation that is designed efficiently predict 16-bit raw audio samples.
The overall computation in the WaveRNN is as follows (biases omitted for brevity):
𝐱ₜ = [𝐜ₜ₋₁,𝐟ₜ₋₁, 𝐜ₜ]
𝐮ₜ = σ(𝐑ᵤ𝐡ₜ₋₁ + 𝐈^*ᵤ𝐱ₜ)
𝐫ₜ = σ(𝐑ᵣ𝐡ₜ₋₁ + 𝐈^*ᵣ𝐱ₜ)
𝐞ₜ = τ(𝐫ₜ ⊙(𝐑ₑ𝐡ₜ₋₁) + 𝐈^*ₑ𝐱ₜ )
𝐡ₜ = 𝐮ₜ ·𝐡ₜ₋₁ + (1-𝐮ₜ) ·𝐞ₜ
𝐲_c, 𝐲_f = split(𝐡ₜ)
P(𝐜ₜ) = softmax(𝐎₂relu(𝐎₁𝐲_c))
P(𝐟ₜ) = softmax(𝐎₄relu(𝐎₃𝐲_f))
where the * indicates a masked matrix whereby the last coarse input 𝐜ₜ is only connected to the fine part of the states 𝐮ₜ, 𝐫ₜ, 𝐞ₜ and 𝐡ₜ and thus only affects the fine output 𝐲_f. The coarse and fine parts 𝐜ₜ and 𝐟ₜ are encoded as scalars in [0, 255] and scaled to the interval [−1, 1]. The matrix 𝐑 formed from the matrices 𝐑ᵤ, 𝐑ᵣ, 𝐑ₑ is computed as a single matrix-vector product to produce the contributions to all three gates 𝐮ₜ, mathbfrₜ and 𝐞ₜ (a variant of the GRU cell. σ and τ are the standard sigmoid and tanh non-linearities.
Each part feeds into a softmax layer over the corresponding 8 bits and the prediction of the 8 fine bits is conditioned on the 8 coarse bits. The resulting Dual Softmax layer allows for efficient prediction of 16-bit samples using two small output spaces (2 8 values each) instead of a single large output space (with 2 16 values).
Papers archive 2025-07-28
26 shown of 26, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Exploratory Evaluation of Speech Content Masking 8 Jan 2024 · 0 repositories · arXiv:2401.03936
-
An End-to-End Multi-Module Audio Deepfake Generation System for ADD Challenge 2023 3 Jul 2023 · 0 repositories · arXiv:2307.00729
-
Evince the artifacts of Spoof Speech by blending Vocal Tract and Voice Source Features 5 Dec 2022 · 0 repositories · arXiv:2212.02013
-
SIMD-size aware weight regularization for fast neural vocoding on CPU 2 Nov 2022 · 0 repositories · arXiv:2211.00898
-
Perfectly Secure Steganography Using Minimum Entropy Coupling 24 Oct 2022 · 2 repositories · arXiv:2210.14889Syntology ran 0 of 4 samples · 4 unverified
-
Adaptive re-calibration of channel-wise features for Adversarial Audio Classification 21 Oct 2022 · 0 repositories · arXiv:2210.11722
-
WaveFit: An Iterative and Non-autoregressive Neural Vocoder based on Fixed-Point Iteration 3 Oct 2022 · 0 repositories · arXiv:2210.01029
-
R-MelNet: Reduced Mel-Spectral Modeling for Neural TTS 30 Jun 2022 · 0 repositories · arXiv:2206.15276
-
NatiQ: An End-to-end Text-to-Speech System for Arabic 15 Jun 2022 · 0 repositories · arXiv:2206.07373
-
VocBench: A Neural Vocoder Benchmark for Speech Synthesis 6 Dec 2021 · 1 repository · arXiv:2112.03099
-
On-device neural speech synthesis 17 Sep 2021 · 0 repositories · arXiv:2109.08710
-
High-Fidelity and Low-Latency Universal Neural Vocoder based on Multiband WaveRNN with Data-Driven Linear Prediction for Discrete Waveform Modeling 20 May 2021 · 1 repository · arXiv:2105.09856
-
Low-Latency Real-Time Non-Parallel Voice Conversion based on Cyclic Variational Autoencoder and Multiband WaveRNN with Data-Driven Linear Prediction 20 May 2021 · 2 repositories · arXiv:2105.09858
-
Enhancing into the codec: Noise Robust Speech Coding with Vector-Quantized Autoencoders 12 Feb 2021 · 0 repositories · arXiv:2102.06610
-
FBWave: Efficient and Scalable Neural Vocoders for Streaming Text-To-Speech on the Edge 25 Nov 2020 · 0 repositories · arXiv:2011.12985
-
TFGAN: Time and Frequency Domain Based Generative Adversarial Network for High-fidelity Speech Synthesis 24 Nov 2020 · 1 repository · arXiv:2011.12206
-
Pretraining Strategies, Waveform Model Choice, and Acoustic Configurations for Multi-Speaker End-to-End Speech Synthesis 10 Nov 2020 · 0 repositories · arXiv:2011.04839
-
Enhancing Speech Intelligibility in Text-To-Speech Synthesis using Speaking Style Conversion 13 Aug 2020 · 1 repository · arXiv:2008.05809Syntology ran 2 of 3 samples · 1 unverified
-
Speaker Conditional WaveRNN: Towards Universal Neural Vocoder for Unseen Speaker and Recording Conditions 9 Aug 2020 · 1 repository · arXiv:2008.05289Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
-
Audiovisual Speech Synthesis using Tacotron2 3 Aug 2020 · 0 repositories · arXiv:2008.00620
-
TTS-Portuguese Corpus: a corpus for speech synthesis in Brazilian Portuguese 11 May 2020 · 1 repository · arXiv:2005.05144
-
Towards Robust Neural Vocoding for Speech Generation: A Survey 5 Dec 2019 · 0 repositories · arXiv:1912.02461
-
A unified sequence-to-sequence front-end model for Mandarin text-to-speech synthesis 11 Nov 2019 · 0 repositories · arXiv:1911.04111
-
DurIAN: Duration Informed Attention Network For Multimodal Synthesis 4 Sep 2019 · 6 repositories · arXiv:1909.01700
-
LPCNet: Improving Neural Speech Synthesis Through Linear Prediction 28 Oct 2018 · 2 repositories · arXiv:1810.11846
-
Efficient Neural Audio Synthesis 23 Feb 2018 · 16 repositories · arXiv:1802.08435Syntology ran 3 of 3 samples · 0 unverified · 1 pointer-only (licence)
Tasks archive 2025-07-28
20 shown of 27 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
| Task | Papers |
|---|---|
| Speech Synthesis | 14 |
| Text to Speech | 11 |
| text-to-speech | 11 |
| CPU | 5 |
| Text-To-Speech Synthesis | 4 |
| GPU | 3 |
| Decoder | 2 |
| Denoising | 2 |
| Face Swapping | 2 |
| Transfer Learning | 2 |
| Voice Conversion | 2 |
| Audio Classification | 1 |
| Audio Synthesis | 1 |
| Automatic Speech Recognition | 1 |
| Automatic Speech Recognition (ASR) | 1 |
| Edge Classification | 1 |
| Face Model | 1 |
| Generative Adversarial Network | 1 |
| Low-latency processing | 1 |
| Polyphone disambiguation | 1 |
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections