Neural Discrete Representation Learning

By Aäron van den Oord, Oriol Vinyals, Koray Kavukcuoglu. 2017

7,634 citations recorded by the source corpus.

Learning useful representations without supervision remains a key challenge in machine learning. In this paper, we propose a simple yet powerful generative model that learns such discrete representations. Our model, the Vector Quantised-Variational AutoEncoder (VQ-VAE), differs from VAEs in two key ways: the encoder network outputs discrete, rather than continuous, codes; and the prior is learnt rather than static. In order to learn a discrete latent representation, we incorporate ideas from vector quantisation (VQ). Using the VQ method allows the model to circumvent issues of "posterior collapse" -- where the latents are ignored when they are paired with a powerful autoregressive decoder -- typically observed in the VAE framework. Pairing these representations with an autoregressive prior, the model can generate high quality images, videos, and speech as well as doing high quality speaker conversion and unsupervised learning of phonemes, providing further evidence of the utility of the learnt representations.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Yoshua Bengio
  2. Mohit Bansal
  3. Jianfeng Gao
  4. Quoc V. Le
  5. Sergey Levine
  6. Percy Liang
  7. Luke S. Zettlemoyer
  8. Ruslan Salakhutdinov

In Representation & Self-Supervised Learning

  1. Yoshua Bengio
  2. James R. Glass
  3. Trevor Darrell
  4. Dacheng Tao
  5. Sergey Levine
  6. Yu-An Chung
  7. Wei-Ning Hsu
  8. Ruslan Salakhutdinov

In Signal Processing

  1. Wei-Ning Hsu
  2. Jinyu Li
  3. Xu Tan
  4. James R. Glass
  5. Junichi Yamagishi
  6. Shinji Watanabe
  7. Dong Yu
  8. Tao Qin

Citations in the corpus