Super-FAN: Integrated Facial Landmark Localization and Super-Resolution of Real-World Low Resolution Faces in Arbitrary Poses with GANs

By Adrian Bulat, Georgios Tzimiropoulos. 2017

336 citations recorded by the source corpus.

This paper addresses 2 challenging tasks: improving the quality of low resolution facial images and accurately locating the facial landmarks on such poor resolution images. To this end, we make the following 5 contributions: (a) we propose Super-FAN: the very first end-to-end system that addresses both tasks simultaneously, i.e. both improves face resolution and detects the facial landmarks. The novelty or Super-FAN lies in incorporating structural information in a GAN-based super-resolution algorithm via integrating a sub-network for face alignment through heatmap regression and optimizing a novel heatmap loss. (b) We illustrate the benefit of training the two networks jointly by reporting good results not only on frontal images (as in prior work) but on the whole spectrum of facial poses, and not only on synthetic low resolution images (as in prior work) but also on real-world images. (c) We improve upon the state-of-the-art in face super-resolution by proposing a new residual-based architecture. (d) Quantitatively, we show large improvement over the state-of-the-art for both face super-resolution and alignment. (e) Qualitatively, we show for the first time good results on real-world low resolution images like the ones of Fig. 1.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Detection and Continual Learning of Novel Face Presentation Attacks (2021, methodology): “Following the aforementioned approach, face landmarks are detected on the visible spectrum using [2].”
To learn image super-resolution, use a GAN to learn how to do image degradation first (2018, methodology): “To our knowledge, the only method that reports face super-resolution results for real-world LR facial images is the very recent work of [28] which presents impressive qualitative results on more than 200 facial images taken from the Widerface dataset [1].”
GAN Prior Embedded Network for Blind Face Restoration in the Wild (2021, methodology): “, up to 200 times downscaling) and q is randomly and uniformly sampled from [5, 50] (i.”
Multispectral Biometrics System Framework: Application to Presentation Attack Detection (2020, neutral): “The face can then be easily detected in the RGB space [46], [47] and the calculated transformation for each camera can be applied to detect the face in the remaining camera frames.”
Deep Learning for Image Super-Resolution: A Survey (2019, methodology): “The Super-FAN [32] and MTUN [164] both introduce FAN to guarantee the consistency of facial landmarks by end-to-end multi-task learning.”
HiFaceGAN: Face Renovation via Collaborative Suppression and Replenishment (2020, methodology): “ional guidance information such as categorial [2] or structural prior [5] to facilitate the replenishment of faithful and photo-realistic details. For blind face restoration [35][6], facial landmarks [4], parsing maps [53], and component heatmaps [59] are typically utilized as external guidance labe”
Towards Real-World Blind Face Restoration with Generative Facial Prior (2021, methodology): “We adopt their official codes except for Super-FAN, for which we use a re-implementation.”
KGSNet: Key-Point-Guided Super-Resolution Network for Pedestrian Detection in the Wild (2020, background): “Thus, the KGSNet can recover ﬁne details of the human body region and can generate a clear SR image from the small-scale one, where super-FAN [29] holds a similar standpoint for face super-resolution.”