How Well Can Generative Adversarial Networks (GAN) Learn Densities: A Nonparametric View

By Tengyuan Liang. 2017

43 citations recorded by the source corpus.

We study in this paper the rate of convergence for learning densities under the Generative Adversarial Networks (GANs) framework, borrowing insights from nonparametric statistics. We introduce an improved GAN estimator that achieves a faster rate, through leveraging the level of smoothness in the target density and the evaluation metric, which in theory remedies the mode collapse problem reported in the literature. A minimax lower bound is constructed to show that when the dimension is large, the exponent in the rate for the new GAN estimator is near optimal. One can view our results as answering in a quantitative way how well GAN learns a wide range of densities with different smoothness properties, under a hierarchy of evaluation metrics. As a byproduct, we also obtain improved bounds for GAN with deeper ReLU discriminator network.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

Citations in the corpus

Deep Neural Networks for Estimation and Inference: Application to Causal Effects and Other Semiparametric Estimands (2018, background): “Deep nets, in the form of Generative Adversarial Networks are a promising tool for density estimation; see Liang (2017) for recent results.”
Interaction Matters: A Note on Non-asymptotic Local Convergence of Generative Adversarial Networks (2018, background): “Recently, several attempts have been made to understand whether GANs learn the target distribution in the statistical sense [Liu et al., 2017, Arora and Zhang, 2017, Liang, 2017, Arora et al., 2017].”
On How Well Generative Adversarial Networks Learn Densities: Nonparametric and Parametric Results (2018, methodology): “The minimax lower bound was ﬁrst proved in the earlier version of this paper (Liang (2017), P. 18-19, for density estimation).”