Deep Unfolding Network for Image Super-Resolution

By Kai Zhang, Luc Van Gool, Radu Timofte. 2020

667 citations recorded by the source corpus.

Learning-based single image super-resolution (SISR) methods are continuously showing superior effectiveness and efficiency over traditional model-based methods, largely due to the end-to-end training. However, different from model-based methods that can handle the SISR problem with different scale factors, blur kernels and noise levels under a unified MAP (maximum a posteriori) framework, learning-based methods generally lack such flexibility. To address this issue, this paper proposes an end-to-end trainable unfolding network which leverages both learningbased methods and model-based methods. Specifically, by unfolding the MAP inference via a half-quadratic splitting algorithm, a fixed number of iterations consisting of alternately solving a data subproblem and a prior subproblem can be obtained. The two subproblems then can be solved with neural modules, resulting in an end-to-end trainable, iterative network. As a result, the proposed network inherits the flexibility of model-based methods to super-resolve blurry, noisy images for different scale factors via a single model, while maintaining the advantages of learning-based methods. Extensive experiments demonstrate the superiority of the proposed deep unfolding network in terms of flexibility, effectiveness and also generalizability.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Designing a Practical Degradation Model for Deep Blind Image Super-Resolution (2021, background): “Specifically, the bicubic degradation can be viewed as a special case of traditional degradation as it can be approximated by setting a proper kernel with zero noise [3, 49].”
Unsupervised Degradation Representation Learning for Blind Super-Resolution (2021, methodology): “To super-resolve real images with unknown degradations, degradation estimation [28, 3] needs to be performed to provide degradation information for non-blind SR networks [42, 33, 40, 34].”
Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (2021, background): “Since the pioneer work SRCNN [9], many CNNbased models have been proposed in recent years [9, 19, 24, 43, 41, 40, 42, 15, 6, 6, 54, 55, 30, 49, 11, 5, 44, 26, 51, 27, 50].”
Learning Continuous Image Representation with Local Implicit Image Function (2020, background): “low-resolution inputs, which is related to the image superresolution tasks [5, 46, 44, 9, 23, 24, 22, 51, 49].”
URetinex-Net: Retinex-based Deep Unfolding Network for Low-light Image Enhancement (2022, methodology): “According to [36], penalty parameters and are expected to iteratively increase for stable convergence.”
Flow-based Kernel Prior with Application to Blind Super-Resolution (2021, methodology): “We compare the proposed DIP-FKP with bicubic interpolation, RCAN [56], DIP [44], Double-DIP [15, 39] and the upper bound model (non-blind USRNet [53] given ground-truth kernels).”
Deep Generalized Unfolding Networks for Image Restoration (2022, methodology): “Moreover, most DUN methods are beset by information loss due to the feature-to-image transformation at the end of each stage.”
ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting (2023, background): “Various SR technologies were explored from different perspectives, including network architecture [31, 32, 33, 34], image prior [35, 36, 37, 38], deep unfolding [39, 40, 41], degradation model [17, 18, 42, 43].”
Unfolding the Alternating Optimization for Blind Super Resolution (2020, methodology): “To make this problem easier to be solved, previous methods [37, 32, 38, 35] usually decompose it into two sequential steps: i) estimating blur kernel from LR image and ii) restoring SR image based on estimated kernel.”
DiffIR: Efficient Diffusion Model for Image Restoration (2023, neutral): “BebyGAN [6] LDM [11] LQ USRGAN [20] DiffIR”