GAN-Based Projector for Faster Recovery With Convergence Guarantees in Linear Inverse Problems

By Ankit Raj, Yuqi Li, Yoram Bresler. 2019

61 citations recorded by the source corpus.

A Generative Adversarial Network (GAN) with generator G trained to model the prior of images has been shown to perform better than sparsity-based regularizers in ill-posed inverse problems. Here, we propose a new method of deploying a GAN-based prior to solve linear inverse problems using projected gradient descent (PGD). Our method learns a network-based projector for use in the PGD algorithm, eliminating expensive computation of the Jacobian of G. Experiments show that our approach provides a speed-up of 60-80x over earlier GAN-based recovery methods along with better accuracy in compressed sensing. Our main theoretical result is that if the measurement matrix is moderately conditioned on the manifold range(G) and the projector is \delta-approximate, then the algorithm is guaranteed to reach O(\delta) reconstruction error in O(log(1/\delta)) steps in the low noise regime. Additionally, we propose a fast method to design such measurement matrices for a given G. Extensive experiments demonstrate the efficacy of this method by requiring 5-10x fewer measurements than random Gaussian measurement matrices for comparable recovery performance. Because the learning of the GAN and projector is decoupled from the measurement operator, our GAN-based projector and recovery algorithm are applicable without retraining to all linear inverse problems in which the measurement operator is moderately conditioned for range(G), as confirmed by experiments on compressed sensing, super-resolution, and inpainting.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Media Technology, Generative Models, Numerical Methods & Scientific Computing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Media Technology

In Generative Models

In Numerical Methods & Scientific Computing

Citations in the corpus

Parallel and Flexible Sampling from Autoregressive Models via Langevin Dynamics (2021, unknown): “These problems have also been studied using learned priors given by GAN’s, with a focus on linear inverse problems (Rick Chang et al., 2017; Bora et al., 2017; Raj et al., 2019).”
Transforming and Projecting Images into Class-conditional Generative Networks (2020, methodology): “The common techniques include: (1) Optimization-based methods: they find the latent vector that can closely reconstruct the input image using gradient-based method (e.g., ADAM, LBFGS) [60, 9, 39, 56, 50, 49, 10] or MCMC [20], (2) Encoder-based methods: they learn an encoder to directly predict the l”
Your Local GAN: Designing Two Dimensional Local Attention Mechanisms for Generative Models (2019, background): “There are, of course numerous other ways to invert, like training an encoder, but also show poor performance on modern GANs with attention. repository: https://github.com/giannisdaras/ylg 2”
Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization (2021, methodology): “Optmization-based methods iteratively optimize a reconstruction objective [99, 92, 56, 1, 36, 16, 75, 74] or perform Markov chain Monte Carlo [21], while encoder-”
EditGAN: High-Precision Semantic Image Editing (2021, methodology): “Previous papers studied encoder-based methods [58, 59, 60, 61, 62], used primarily optimization-based techniques [63, 64, 65, 66, 67, 68, 69, 26], and developed hybrid approaches [63, 24, 25, 70, 71].”