Blind Super-Resolution With Iterative Kernel Correction

By Jinjin Gu, Hannan Lu, Wangmeng Zuo, Chao Dong. 2019

571 citations recorded by the source corpus.

Deep learning based methods have dominated super-resolution (SR) field due to their remarkable performance in terms of effectiveness and efficiency. Most of these methods assume that the blur kernel during downsampling is predefined/known (e.g., bicubic). However, the blur kernels involved in real applications are complicated and unknown, resulting in severe performance drop for the advanced SR methods. In this paper, we propose an Iterative Kernel Correction (IKC) method for blur kernel estimation in blind SR problem, where the blur kernels are unknown. We draw the observation that kernel mismatch could bring regular artifacts (either over-sharpening or over-smoothing), which can be applied to correct inaccurate blur kernels. Thus we introduce an iterative correction scheme -- IKC that achieves better results than direct kernel estimation. We further propose an effective SR network architecture using spatial feature transform (SFT) layers to handle multiple blur kernels, named SFTMD. Extensive experiments on synthetic and real-world images show that the proposed IKC method with SFTMD can provide visually favorable SR results and the state-of-the-art performance in blind SR problem.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Radu Timofte
3. Chen Change Loy
4. Lei Zhang
5. Ming-Hsuan Yang
6. Chao Dong
7. Yu Qiao
8. Wangmeng Zuo

In Media Technology

1. Radu Timofte
2. Luc Van Gool
3. Kai Zhang
4. Chao Dong
5. Lei Zhang
6. Jingyun Liang
7. Wangmeng Zuo
8. Jinjin Gu

Citations in the corpus

• Designing a Practical Degradation Model for Deep Blind Image Super-Resolution (2021, methodology): “As a further step, some blind methods propose to fuse the kernel estimation into the network design [16, 31].”
• Unsupervised Degradation Representation Learning for Blind Super-Resolution (2021, methodology): “However, these methods suffer a severe performance drop when the real degradation differs from their assumption [12].”
• Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization (2023, methodology): “In the early study, bicubic downsampling or some simple degradations (Dong et al. 2014; Lai et al. 2017; Gu et al. 2019) Ecker, and Bethge 2015) presented an optimization based method to transfer the style of a given artwork to a content image.”
• Meta-Transfer Learning for Zero-Shot Super-Resolution (2020, methodology): “IKC [11] is trained for bicubic subsampling, it never sees aliased images during training.”
• Blind image super-resolution with spatially variant degradations (2019, background): “In such complex setting, the initial kernel estimation step [Gu et al. 2019] becomes more challenging.”
• Real-World Single Image Super-Resolution: A Brief Review (2021, methodology): “DAN [92] consists of a chain of alternately stacked restorers and estimators, which are responsible for the restoration of the HR image and the estimation of the blur kernel, respectively.”
• PIPAL: a Large-Scale Image Quality Assessment Dataset for Perceptual Image Restoration (2020, background): “Since the pioneer work of using Convolution Neural Networks (CNNs) to learn the IR mappings [21,12], the deep-learning-based algorithms have dominated IR research due to their remarkable performance and usability [16,19].”
• RankSRGAN: Generative Adversarial Networks With Ranker for Image Super-Resolution (2019, methodology): “[10] first introduced convolutional neural networks (CNNs) to the SR task, a series of learning-based works [40, 15, 20, 16, 13, 14] have achieved great improvements in terms of PSNR.”
• Unsupervised Learning for Real-World Super-Resolution (2019, background): “Early attempts [4] to this problem include explicitly estimating the unknown point spread function itself [28, 13].”
• Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild (2024, background): “However, these methods are often based on specific degradation assumptions [25, 50, 56] and therefore lack generalization ability to other degradations [29, 52, 94].”
• Real-World Super-Resolution via Kernel Estimation and Noise Injection (2020, background): “We first analyze the impact of different kernels on the downsampled image [42, 49, 11].”
• Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data (2021, methodology): “Classical degradation model [10, 28] is widely adopted in blind SR methods [54, 15, 33].”
• Blind Super-Resolution Kernel Estimation using an Internal-GAN (2019, methodology): “A work concurrent to ours, IKC [10], performs Blind-SR by alternating between the kernel estimation and the SR image reconstruction.”
• Unfolding the Alternating Optimization for Blind Super Resolution (2020, methodology): “Following the setting in [14], the kernel size is set as 21.”
• NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results (2020, background): “The research direction of blind super-resolution [49, 20, 6] does not fully address this setting since it often relies on paired data and constrained image formation models.”
• Deep Unfolding Network for Image Super-Resolution (2020, methodology): “The compared methods include RCAN [70], ZSSR [51], IKC [20] and IRCNN [65].”