Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration

By Yunjin Chen, Thomas Pock. 2015

1,323 citations recorded by the source corpus.

Image restoration is a long-standing problem in low-level computer vision with many interesting applications. We describe a flexible learning framework based on the concept of nonlinear reaction diffusion models for various image restoration problems. By embodying recent improvements in nonlinear diffusion models, we propose a dynamic nonlinear reaction diffusion model with time-dependent parameters (i.e., linear filters and influence functions). In contrast to previous nonlinear diffusion models, all the parameters, including the filters and the influence functions, are simultaneously learned from training data through a loss based approach. We call this approach TNRD—Trainable Nonlinear Reaction Diffusion. The TNRD approach is applicable for a variety of image restoration tasks by incorporating appropriate reaction force. We demonstrate its capabilities with three representative applications, Gaussian image denoising, single image super resolution and JPEG deblocking. Experiments show that our trained nonlinear diffusion models largely benefit from the training of the parameters and finally lead to the best reported performance on common test datasets for the tested applications. Our trained models preserve the structural simplicity of diffusion models and take only a small number of diffusion steps, thus are highly efficient. Moreover, they are also well-suited for parallel computation on GPUs, which makes the inference procedure extremely fast.

Subfields: Atomic and Molecular Physics, and Optics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Atomic and Molecular Physics, and Optics

In Computer Vision and Pattern Recognition

Citations in the corpus

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks (2016, methodology): “In some cases, this refinement can be approximated by neural networks: Chen & Pock [9] formulate their reaction diffusion model as a CNN and apply it to image denoising, deblocking and superresolution.”
Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network (2016, methodology): “In this section, we show ESPCN trained on ImageNet compared to results from SRCNN [7] and the TNRD [3] which is currently the best performing approach published.”
Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser (2020, neutral): “Current state-of-the-art denoising results using CNNs are far superior, both numerically and visually, to results of previous methods [27–29].”
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising (2016, methodology): “In [16], a trainable nonlinear reaction diffusion (TNRD) model was proposed and it can be expressed as a feed-forward deep network by unfolding a fixed number of gradient descent inference steps.”
Non-Local Recurrent Network for Image Restoration (2018, methodology): “Image Denoising: For a fair comparison with other methods based on deep networks, we train our model under two settings: (1) We use the training data as in TNRD [7], DnCNN [49] and NLNet [23], and the result is shown in Table 4.”
Learning to Optimize: A Primer and A Benchmark (2021, background): “Popular application examples include denoising [63; 172; 256; 51; 192; 126; 234; 134; 142], deblurring [63; 256; 146; 127; 58; 142; 162; 257; 121], super-resolution [175; 72; 172; 256; 51; 127; 257], inpainting [175; 172; 192; 3], and compressive sensing [175; 153; 43; 63; 171; 141; 255; 98; 142], J”
A High-Quality Denoising Dataset for Smartphone Cameras (2018, methodology): “Applied/ Evaluated BM3D [10] NLM [4] KSVD [1] KSVDDCT [11] KSVDG [11] LPGPCA [32] FoE [27] MLP [6] WNNM [16] GLIDE [29] TNRD [8] EPLL [35] DnCNN [33]”
Universal Denoising Networks : A Novel CNN Architecture for Image Denoising (2017, methodology): “We emphasize that as opposed to previous networks [6, 23, 38] that followed a similar unrolling strategy, our network architecture is based on a constrained minimization formulation (5) rather than on an unconstrained one (4).”
Learning Deep CNN Denoiser Prior for Image Restoration (2017, methodology): “On the contrary, discriminative learning methods try to learn the prior parameters Θ and a compact inference through an optimization of a loss function on a training set containing degraded-clean image pairs [2, 13, 51, 55, 57].”
Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing (2019, methodology): “[16] 2017 Computational imaging Image denoising, JPEG deblocking Nonlinear diffusion Jin et al.”
Self2Self With Dropout: Learning Self-Supervised Denoising From Single Image (2020, neutral): “[23, 9, 30].”
Residual Dense Network for Image Restoration (2018, methodology): “we mainly take image SR as an example and give specific illustrations for image DN and CAR cases.”
NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results (2020, background): “The generator structure can also be described as the generalization of one stage TNRD [11] and UDNet [42] that have good reconstruction performance for image denoising problem.”