WaveNet: A Generative Model for Raw Audio

By Aäron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew W. Senior, Koray Kavukcuoglu. 2016

8,319 citations recorded by the source corpus.

This paper introduces WaveNet, a deep neural network for generating raw audio waveforms. The model is fully probabilistic and autoregressive, with the predictive distribution for each audio sample conditioned on all previous ones; nonetheless we show that it can be efficiently trained on data with tens of thousands of samples per second of audio. When applied to text-to-speech, it yields state-of-the-art performance, with human listeners rating it as significantly more natural sounding than the best parametric and concatenative systems for both English and Mandarin. A single WaveNet can capture the characteristics of many different speakers with equal fidelity, and can switch between them by conditioning on the speaker identity. When trained to model music, we find that it generates novel and often highly realistic musical fragments. We also show that it can be employed as a discriminative model, returning promising results for phoneme recognition.

Subfields: Computer Vision and Pattern Recognition, Generative Models, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Ming-Hsuan Yang
  5. Jitendra Malik
  6. Xiaogang Wang
  7. Andrew Zisserman
  8. Andrea Vedaldi

In Generative Models

  1. Stefano Ermon
  2. Arash Vahdat
  3. Yoshua Bengio
  4. Diederik P. Kingma
  5. Jiaming Song
  6. Jonathan Ho
  7. Jun-Yan Zhu
  8. Aaron C. Courville

In Speech & Audio

  1. Wei-Ning Hsu
  2. Jinyu Li
  3. Xu Tan
  4. Yu Zhang
  5. Junichi Yamagishi
  6. James R. Glass
  7. Hung-yi Lee
  8. Lei He

In Signal Processing

  1. Junichi Yamagishi
  2. Jinyu Li
  3. Xu Tan
  4. Dong Yu
  5. Tao Qin
  6. Frank K. Soong
  7. Yoshua Bengio
  8. Wei-Ning Hsu

Citations in the corpus