Invertible generative models for inverse problems: mitigating representation error and dataset bias

By Muhammad Asim, Ali Ahmed, Paul Hand. 2019

174 citations recorded by the source corpus.

Trained generative models have shown remarkable performance as priors for inverse problems in imaging -- for example, Generative Adversarial Network priors permit recovery of test images from 5-10x fewer measurements than sparsity priors. Unfortunately, these models may be unable to represent any particular image because of architectural choices, mode collapse, and bias in the training dataset. In this paper, we demonstrate that invertible neural networks, which have zero representation error by design, can be effective natural signal priors at inverse problems such as denoising, compressive sensing, and inpainting. Given a trained generative model, we study the empirical risk formulation of the desired inverse problem under a regularization that promotes high likelihood images, either directly by penalization or algorithmically by initialization. For compressive sensing, invertible priors can yield higher accuracy than sparsity priors across almost all undersampling ratios, and due to their lack of representation error, invertible priors can yield better reconstructions than GAN priors for images that have rare features of variation within the biased training set, including out-of-distribution natural images. We additionally compare performance for compressive sensing to unlearned methods, such as the deep decoder, and we establish theoretical bounds on expected recovery error in the case of a linear invertible model.

Subfields: Computer Vision and Pattern Recognition, Generative Models, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

In Signal Processing

Citations in the corpus

D2C: Diffusion-Decoding Models for Few-Shot Conditional Generation (2021, background): “These properties c include (but are not limited to2) labels [61], text descriptions [60, 76], noisy or partial observations [11, 5, 47, 24], and manipulation constraints [69].”
Intermediate Layer Optimization for Inverse Problems using Deep Generative Models (2021, background): “Solving inverse problems for out-of-distribution images has been studied in (Asim et al., 2020).”
Large-Scale Generative Data-Free Distillation (2020, background): “llowing layer. The initial motivation was to save memory by computing the activations on-the-ﬂy during backpropagation, while later researchers have also discovered its potential for image generation [2]. We stress here, that while GAN and GAN-like methods generate very realistic and high-quality im”
Prior Image-Constrained Reconstruction using Style-Based Generative Models (2021, unknown): “(6) is formulated as the following optimization problem: Although the above problem is non-convex, similar to previous works (Bora et al., 2017; Asim et al., 2020), useful estimates ˆ f can be obtained by iterative gradient descent-based optimization.”
Robust Compressed Sensing MRI with Deep Generative Priors (2021, methodology): “[6] Muhammad Asim, Ali Ahmed, and Paul Hand.”
Score-based diffusion models for accelerated MRI (2021, methodology): “Before the recent surge of diffusion models, there have been several approahces to use generative models, especially generative adversarial networks (GAN) (Goodfellow et al., 2014) as priors for solving inverse problems (Marinescu et al., 2021; Asim et al., 2020).”