When and How Can Deep Generative Models be Inverted?

By Aviad Aberdam, Dror Simon, Michael Elad. 2020

13 citations recorded by the source corpus.

Deep generative models (e.g. GANs and VAEs) have been developed quite extensively in recent years. Lately, there has been an increased interest in the inversion of such a model, i.e. given a (possibly corrupted) signal, we wish to recover the latent vector that generated it. Building upon sparse representation theory, we define conditions that are applicable to any inversion algorithm (gradient descent, deep encoder, etc.), under which such generative models are invertible with a unique solution. Importantly, the proposed analysis is applicable to any trained model, and does not depend on Gaussian i.i.d. weights. Furthermore, we introduce two layer-wise inversion pursuit algorithms for trained generative networks of arbitrary depth, and accompany these with recovery guarantees. Finally, we validate our theoretical results numerically and show that our method outperforms gradient descent when inverting such generators, both for clean and corrupted signals.

Subfields: Computational Mechanics, Oceanography, Generative Models, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Bolei Zhou
2. Vardan Papyan
3. Ming-Hsuan Yang
4. David L. Donoho
5. Jing-Hao Xue
6. Jan Kautz
7. Yulun Zhang
8. José Manuel Álvarez

In Oceanography

1. David L. Donoho
2. Jan Kautz
3. Pavlo Molchanov

In Generative Models

1. Yujiu Yang
2. Weihao Xia
3. Jing-Hao Xue
4. Ming-Hsuan Yang
5. Bolei Zhou
6. Yulun Zhang
7. Daniel Cohen-Or
8. Hongxu Yin

In Signal Processing

1. Bolei Zhou
2. David L. Donoho
3. Ming-Hsuan Yang
4. Jing-Hao Xue
5. Hongxu Yin
6. Jan Kautz
7. Yulun Zhang
8. Vardan Papyan

Citations in the corpus

• GAN Inversion: A Survey (2021, methodology): “Recent methods aim to address this issue from different theoretical perspectives, such as compressed sensing [82], [83], [84] and sparse representation [85].”
• Prevalence of neural collapse during the terminal phase of deep learning training (2020, background): “In the deepnet literature, the authors of (44–48) studied the stability of deepnets to adversarial examples.”
• Deep Neural Networks are Surprisingly Reversible: A Baseline for Zero-Shot Inversion (2021, background): “For example, one direction of works provides theoretical proof of the invertibility assuming the weights of DNNs are random-like [19, 29, 30], and expansion rates are larger than a certain constant [21, 31].”