The Expressive Power of a Class of Normalizing Flow Models

By Zhifeng Kong, Kamalika Chaudhuri. 2020

58 citations recorded by the source corpus.

Normalizing flows have received a great deal of recent attention as they allow flexible generative modeling as well as easy likelihood computation. While a wide variety of flow models have been proposed, there is little formal understanding of the representation power of these models. In this work, we study some basic normalizing flows and rigorously establish bounds on their expressive power. Our results indicate that while these flows are highly expressive in one dimension, in higher dimensions their representation power may be limited, especially when the flows have moderate depth.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Generative Models, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Generative Models

In Probabilistic & Bayesian Methods

Citations in the corpus

Score-based Generative Modeling in Latent Space (2021, background): “However, previous works found that augmenting neural ODEs [16, 17] and more generally generative models [18, 19, 20, 21] with latent variables improves their expressivity.”
VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models (2020, background): “VAEs also do not suffer from expressivity limitations that normalizing flows face (Dupont et al., 2019; Kong & Chaudhuri, 2020), and in fact, they have recently shown state-of-the-art generative results among non-autoregressive likelihood-based models (Vahdat & Kautz, 2020).”