Recurrent Inference Machines for Solving Inverse Problems

By Patrick Putzky, Max Welling. 2017

134 citations recorded by the source corpus.

Much of the recent research on solving iterative inference problems focuses on moving away from hand-chosen inference algorithms and towards learned inference. In the latter, the inference process is unrolled in time and interpreted as a recurrent neural network (RNN) which allows for joint learning of model and inference parameters with back-propagation through time. In this framework, the RNN architecture is directly derived from a hand-chosen inference algorithm, effectively limiting its capabilities. We propose a learning framework, called Recurrent Inference Machines (RIM), in which we turn algorithm construction the other way round: Given data and a task, train an RNN to learn an inference algorithm. Because RNNs are Turing complete [1, 2] they are capable to implement any inference algorithm. The framework allows for an abstraction which removes the need for domain knowledge. We demonstrate in several image restoration experiments that this abstraction is effective, allowing us to achieve state-of-the-art performance on image denoising and super-resolution tasks and superior across-task generalization.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Generalization in anti-causal learning (2018, methodology): “Experiments: Eslami et al. (2016); Putzky and Welling (2017) use an RNN to iteratively reﬁne inference over a known generative model of the data.”
Iterative Amortized Inference (2018, background): “(Putzky & Welling, 2017) use recurrent inference models for MAP estimation of denoised images in linear models.”
Recurrent machines for likelihood-free inference (2018, background): “…that automatically learning neural optimizers from data is possible, achieving results close to the state-of-the-art for the task of training neural networks (Andrychowicz et al., 2016) or solving inverse problems (Putzky and Welling, 2017) when the gradient of the objective function is available.”
MoDL: Model-Based Deep Learning Architecture for Inverse Problems (2017, methodology): “In summary, the main novelties of the proposed scheme over related deep-learning schemes described above are (a) the architecture involving numerical optimization blocks within the deep network, which enables the easy use of complex forward models and additional image priors [24], while offering bet”
CNN-Based Projected Gradient Descent for Consistent CT Image Reconstruction (2017, methodology): “In [24], the iterative algorithm is replaced by a recurrent neural network (RNN).”
Solving inverse problems using data-driven models (2019, background): “Quinto (1993), ‘Singularities of the X-ray transform and limited data tomography in R(2) and R(3)’, SIAM J.”
Solving ill-posed inverse problems using iterative deep neural networks (2017, methodology): “This paper develops a framework for learned iterative reconstruction, generalising the ideas in [33] along several directions.”
Model-Based Learning for Accelerated, Limited-View 3-D Photoacoustic Tomography (2017, methodology): “Inspired by [44] and [45] we take the structure of (5) as a starting point: Each iteration consists of updating xk by combining measurement information delivered through the gradient ∇d(y, Axk) with an image processing step.”
Learning to Optimize: A Primer and A Benchmark (2021, unknown): “Popular application examples include denoising [63; 172; 256; 51; 192; 126; 234; 134; 142], deblurring [63; 256; 146; 127; 58; 142; 162; 257; 121], super-resolution [175; 72; 172; 256; 51; 127; 257], inpainting [175; 172; 192; 3], and compressive sensing [175; 153; 43; 63; 171; 141; 255; 98; 142], J”