Variational Inference with Normalizing Flows

By Danilo Jimenez Rezende, Shakir Mohamed. 2015

5,033 citations recorded by the source corpus.

The choice of approximate posterior distribution is one of the core problems in variational inference. Most applications of variational inference employ simple families of posterior approximations in order to allow for efficient inference, focusing on mean-field or other simple structured approximations. This restriction has a significant impact on the quality of inferences made using variational methods. We introduce a new approach for specifying flexible, arbitrarily complex and scalable approximate posterior distributions. Our approximations are distributions constructed through a normalizing flow, whereby a simple initial density is transformed into a more complex one by applying a sequence of invertible transformations until a desired level of complexity is attained. We use this view of normalizing flows to develop categories of finite and infinitesimal flows and provide a unified view of approaches for constructing rich posterior approximations. We demonstrate that the theoretical advantages of having posteriors that better match the true posterior, combined with the scalability of amortized variational approaches, provides a clear improvement in performance and applicability of variational inference.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Generative Models, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Yoshua Bengio
  2. Sergey Levine
  3. Mohit Bansal
  4. Quoc V. Le
  5. Ruslan Salakhutdinov
  6. Jianfeng Gao
  7. Percy Liang
  8. Kyunghyun Cho

In Statistical and Nonlinear Physics

  1. Stefano Ermon
  2. Arash Vahdat
  3. Yang Song
  4. Yoshua Bengio
  5. Jun Zhu
  6. Diederik P. Kingma
  7. Jiaming Song
  8. Jonathan Ho

In Generative Models

  1. Stefano Ermon
  2. Arash Vahdat
  3. Yoshua Bengio
  4. Diederik P. Kingma
  5. Jiaming Song
  6. Aaron C. Courville
  7. Jan Kautz
  8. Jonathan Ho

In Probabilistic & Bayesian Methods

  1. Stefano Ermon
  2. Yoshua Bengio
  3. Max Welling
  4. David M. Blei
  5. Diederik P. Kingma
  6. Richard E. Turner
  7. Jun Zhu
  8. Yang Song

Citations in the corpus