Blind Face Restoration via Deep Multi-scale Component Dictionaries

By Xiaoming Li, Chaofeng Chen, Shangchen Zhou, Xianhui Lin, Wangmeng Zuo, Lei Zhang. 2020

228 citations recorded by the source corpus.

Recent reference-based face restoration methods have received considerable attention due to their great capability in recovering high-frequency details on real low-quality images. However, most of these methods require a high-quality reference image of the same identity, making them only applicable in limited scenes. To address this issue, this paper suggests a deep face dictionary network (termed as DFDNet) to guide the restoration process of degraded observations. To begin with, we use K-means to generate deep dictionaries for perceptually significant face components (\ie, left/right eyes, nose and mouth) from high-quality images. Next, with the degraded input, we match and select the most similar component features from their corresponding dictionaries and transfer the high-quality details to the input via the proposed dictionary feature transfer (DFT) block. In particular, component AdaIN is leveraged to eliminate the style diversity between the input and dictionary features (\eg, illumination), and a confidence score is proposed to adaptively fuse the dictionary feature to the input. Finally, multi-scale dictionaries are adopted in a progressive manner to enable the coarse-to-fine restoration. Experiments show that our proposed method can achieve plausible performance in both quantitative and qualitative evaluation, and more importantly, can generate realistic and promising results on real degraded images without requiring an identity-belonging reference. The source code and models are available at \url{this https URL}.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Chen Change Loy
2. Luc Van Gool
3. Yu Qiao
4. Lei Zhang
5. Chao Dong
6. Ming-Hsuan Yang
7. Dacheng Tao
8. Xiaogang Wang

In Generative Models

1. Chao Dong
2. Chen Change Loy
3. Lei Zhang
4. Ying Shan
5. Peiran Ren
6. Tao Yang
7. Xuansong Xie
8. Xintao Wang

Citations in the corpus

• Image Super-Resolution via Iterative Refinement (2021, neutral): “Numerous super-resolution methods have been proposed [16], [30], [31], [32], [49], [50], [51], [52], [53].”
• DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior (2023, background): “facial parsing maps [5], facial landmarks[9; 27], and facial component heatmaps [62]) or reference priors[34; 33; 32; 13] as auxiliary information to guide the face restoration process.”
• Towards Robust Blind Face Restoration with Codebook Lookup Transformer (2022, unknown): “Reference-based approaches [9, 24–26] have been proposed to circumvent the above limitations.”
• GAN Prior Embedded Network for Blind Face Restoration in the Wild (2021, methodology): “(a) Degraded faces; (b) Super-FAN [5]; (c) GFRNet [29]; (d) GWAInet [9]; (e) Pix2PixHD [43]; (f) DFDNet [27]; (g) HiFaceGAN [47]; (h) GPEN; (i) Ground truth.”
• Towards Real-World Blind Face Restoration with Generative Facial Prior (2021, methodology): “We compare our GFP-GAN with several state-of-the-art face restoration methods: HiFaceGAN [69], DFDNet [46], PSFRGAN [6], Super-FAN [4] and Wan et al. [63].”
• GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution (2020, methodology): “For DFDNet, we evaluate the performance on LR images with unknown degradations4.”
• Low-Light Image and Video Enhancement Using Deep Learning: A Survey (2021, background): “Similar work has been done on image super-resolution [106], [107] and face restoration [108].”