Fast Wavenet Generation Algorithm

By T. Paine, Pooya Khorrami, Shiyu Chang, Yang Zhang, Prajit Ramachandran, Mark Hasegawa-Johnson, Thomas S. Huang. 2016

88 citations recorded by the source corpus.

This paper presents an efficient implementation of the Wavenet generation process called Fast Wavenet. Compared to a naive implementation that has complexity O(2^L) (L denotes the number of layers in the network), our proposed approach removes redundant convolution operations by caching previous calculations, thereby reducing the complexity to O(L) time. Timing experiments show significant advantages of our fast implementation over a naive one. While this method is presented for Wavenet, the same scheme can be applied anytime one wants to perform autoregressive generation or online prediction using a model with dilated convolution layers. The code for our method is publicly available.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

In Speech & Audio

In Signal Processing

Citations in the corpus

Deep Voice: Real-time Neural Text-to-Speech (2017, background): “We answer this question in the afﬁrmative and demonstrate efﬁcient, faster-than-real-time WaveNet inference kernels that produce high-quality 16 kHz audio and realize a 400X speedup over previous WaveNet inference implementations (Paine et al., 2016).”
A Survey on Neural Speech Synthesis (2021, background): “Therefore, a lot of works [250, 114, 227] investigate lightweight and fast vocoders.”
Hierarchical Autoregressive Image Models with Auxiliary Decoders (2019, methodology): “We use a version of incremental sampling (Paine et al., 2016) which uses buffers to avoid unnecessary recomputation.”
GANSynth: Adversarial Neural Audio Synthesis (2019, background): “Due to their high quality, a lot of research has gone into speeding up generation, but the methods introduce signiﬁcant overhead such as training a secondary student network or writing highly customized low-level kernels (van den Oord et al., 2018; Paine et al., 2016).”
WaveFlow: A Compact Flow-based Model for Raw Audio (2019, methodology): “We implement convolution queue (Paine et al., 2016) in Python to cache the intermediate hidden states in WaveFlow for autoregressive inference over the height dimension, which brings another 3× to 5× speedup depending on height h.”
Neural Source-Filter Waveform Models for Statistical Parametric Speech Synthesis (2019, methodology): “7Unlike naive open-source implementations, our WaveNet-vocoder avoids redundant CONV operations as the so-called Fast-WaveNet did [59].”
HiLLoC: Lossless Image Compression with Hierarchical Latent Variable Models (2019, background): “This effect can be mitigated by using dynamic programming (Le Paine et al., 2016; Ramachandran et al., 2017), and altering model architecture (Reed et al., 2017), but on parallel architectures sampling/decompression is still significantly slower than with VAE models.”
Wasserstein GAN and Waveform Loss-Based Acoustic Model Training for Multi-Speaker Text-to-Speech Synthesis Systems Using a WaveNet Vocoder (2018, methodology): “To implement the WaveNet model, we referenced [36] and adopted a modified version of the WaveNet architecture.”
MidiNet: A Convolutional Generative Adversarial Network for Symbolic-Domain Music Generation (2017, methodology): “This includes models for generating a melody sequence or audio waveforms by following a few priming notes [10,18,19,22,31,33], accompanying a melody sequence with music of other parts [15], or playing a duet with human [4,26].”
Fftnet: A Real-Time Speaker-Dependent Neural Vocoder (2018, methodology): “[13] introduce an algorithmic improvement for the same architecture called Fast WaveNet, which can synthesize a second of audio in a roughly a minute.”
Speech Enhancement Using Bayesian Wavenet (2017, methodology): “(10) Obtaining the previous inferred value x̂ t−τ1:t−1 can be implemented efficiently using the method in [24].”