Learning to Super-Resolve Blurry Face and Text Images

By Xiangyu Xu, Deqing Sun, Jin-shan Pan, Yujin Zhang, Hanspeter Pfister, Ming-Hsuan Yang. 2017

223 citations recorded by the source corpus.

We present an algorithm to directly restore a clear highresolution image from a blurry low-resolution input. This problem is highly ill-posed and the basic assumptions for existing super-resolution methods (requiring clear input) and deblurring methods (requiring high-resolution input) no longer hold. We focus on face and text images and adopt a generative adversarial network (GAN) to learn a category-specific prior to solve this problem. However, the basic GAN formulation does not generate realistic highresolution images. In this work, we introduce novel training losses that help recover fine details. We also present a multi-class GAN that can process multi-class image restoration tasks, i.e., face and text images, using a single generator network. Extensive experiments demonstrate that our method performs favorably against the state-of-the-art methods on both synthetic and real-world images at a lower computational cost.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Recovering Realistic Texture in Image Super-Resolution by Deep Spatial Feature Transform (2018, background): “[48] but they only focus on two classes of images, i.”
Deep Image Deblurring: A Survey (2022, background): “In addition to these main types of blur there can be other causes, such as channel-dependent blur resulting from chromatic aberration (Son and Park 2011; Sun et al. 2017).”
Deep Learning for Image Super-Resolution: A Survey (2019, methodology): “[63] propose a multi-class GAN-based FH model composed of a generic generator and class-specific discriminators.”
Dynamic Scene Deblurring Using Spatially Variant Recurrent Neural Networks (2018, background): “Recently, deep learning has been widely used in many low-level vision problems, such as denoising [1, 20, 45], super-resolution[4, 35, 13, 14, 15, 43, 31], dehazing [27], derain/dedirt [6, 5], edge-preserving filtering [39, 18], and image deblurring (non-blind [28, 38, 44] and blind [29, 2, 42]).”
Detecting small faces in the wild based on generative adversarial network and contextual information (2019, methodology): “Moreover, images are often zoomed in with a small upscaling factor (2 × at most) in [8] , otherwise, artifacts will be generated.”
Finding Tiny Faces in the Wild with Generative Adversarial Network (2018, methodology): “To detect small faces, [28] directly up-samples images using bi-linear operation and exhaustively searches faces on the up-sampled images.”
Gated Fusion Network for Single Image Dehazing (2018, methodology): “Recently, CNNs have also been used for image recovering problems [4, 14, 39, 42, 43, 44].”
Exploiting Diffusion Prior for Real-World Image Super-Resolution (2023, unknown): “Due to the lack of real-world paired data for training, some methods [18, 39, 59, 61, 68, 76] propose to implicitly learn a degradation model from LR images in an unsupervised manner such as Cycle-GAN [86] and contrastive learning [45].”
Towards Open-World Text-Guided Face Image Generation and Manipulation (2021, background): “, label [20], [21], sketch [22], [23], text [24], [25], or control vector like gaze [26] or light direction [27].”
Single Image Dehazing via Conditional Generative Adversarial Network (2018, methodology): “We also note that several methods develop GANs [39] to solve image deraining and image super-resolution [36].”
Human-Aware Motion Deblurring (2019, background): “In a similar spirit, more advanced deep learning based deblurring models [35, 54, 27, 21, 58, 40] have been designed, e.”
KGSNet: Key-Point-Guided Super-Resolution Network for Pedestrian Detection in the Wild (2020, background): “However, as we all know, the solution to the MSE optimization problem usually lacks high-frequency details, which results in blurred images. b) Body structure loss: As mentioned in [73] and [74], only using pixelwise MSE loss will produce images with oversmooth texture.”
TediGAN: Text-Guided Diverse Image Generation and Manipulation (2020, background): “, label [64, 63], sketch [65, 16], text [53, 36], or control vector like gaze [13]”