Style Aggregated Network for Facial Landmark Detection

By Xuanyi Dong, Yan Yan, Wanli Ouyang, Yi Yang. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

324 citations recorded by the source corpus.

Recent advances in facial landmark detection achieve success by learning discriminative features from rich deformation of face shapes and poses. Besides the variance of faces themselves, the intrinsic variance of image styles, e.g., grayscale vs. color images, light vs. dark, intense vs. dull, and so on, has constantly been overlooked. This issue becomes inevitable as increasing web images are collected from various sources for training neural networks. In this work, we propose a style-aggregated approach to deal with the large intrinsic variance of image styles for facial landmark detection. Our method transforms original face images to style-aggregated images by a generative adversarial module. The proposed scheme uses the style-aggregated image to maintain face images that are more robust to environmental changes. Then the original face images accompanying with style-aggregated ones play a duet to train a landmark detector which is complementary to each other. In this way, for each face, our method takes two images as input, i.e., one in its original style and the other in the aggregated style. In experiments, we observe that the large variance of image styles would degenerate the performance of facial landmark detectors. Moreover, we show the robustness of our method to the large variance of image styles by comparing to a variant of our approach, in which the generative adversarial module is removed, and no style-aggregated images are used. Our approach is demonstrated to perform well when compared with state-of-the-art algorithms on benchmark datasets AFLW and 300-W. Code is publicly available on GitHub: https://github.com/D-X-Y/SAN

Subfields: Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/77875d6e4d8c7ed3baeb259fd5696e921f59d7ad

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

High-Resolution Representations for Labeling Pixels and Regions (2019, unknown): “SAN [24] ResNet-152 1.”
Teacher Supervises Students How to Learn From Partially Labeled Images for Facial Landmark Detection (2019, methodology): “Normalized Mean Error (NME) is usually applied to evaluate the performance for facial landmark predictions [27, 34, 47, 9].”
Very Long Natural Scenery Image Prediction by Outpainting (2019, methodology): “After “Pix2Pix” [12] framework, which use a conditional adversarial network [5] to learn a mapping from input to output images, similar ideas have been applied to related tasks, such as translating sketches to photographs [21], style translation [35, 4], etc.”
Camera Style Adaptation for Person Re-identification (2017, background): “Recently, GANs have also been applied to image-to-image translation [13, 51, 22], style transfer [8, 14, 6] and cross domain image generation [2, 31, 5].”
The face of art (2019, methodology): “In Table 1 we compare our approach with recently proposed state of-the-art algorithms ([Dong et al. 2018], [Kowalski et al. 2017], [Zhang et al. 2018]) on natural face images.”
Deep High-Resolution Representation Learning for Visual Recognition (2019, unknown): “14 - SAN [34] ResNet-152”
Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression (2019, unknown): “…4.55 Wing CVPR 18 [17] 3.27 7.18 4.04 AWing(Ours) 3.77 6.52 4.31 Inter-ocular Normalization PCD-CNN CVPR 18 [30] 3.67 7.62 4.44 CPM+SBR CVPR 18 [14] 3.28 7.58 4.10 SAN CVPR 18 [14] 3.34 6.60 3.98 LAB CVPR 18 [58] 2.98 5.19 3.49 DU-Net ECCV 18 [52] 2 [54] 5.05 45.32 6.80 DAN CVPRW17 [29] 4.30…”
Supervision-by-Registration: An Unsupervised Approach to Improve the Precision of Facial Landmark Detectors (2018, methodology): “Though image-based facial landmark detectors [8, 19, 37, 38, 3, 20] can achieve very good performance on images, sequentially running these detectors on each frame of a video in a tracking-by-detection fashion usually leads to jittering and unstable detections.”
Generalizing a Person Retrieval Model Hetero- and Homogeneously (2018, background): “Beneﬁt from the remarkable success of deep learning [24,25,26,27,28], recent works [2,29,30] attempt to predict pseudo-labels to unlabeled samples based on the deep learning framework.”
A Bottom-Up Clustering Approach to Unsupervised Person Re-Identification (2019, methodology): “In recent years, some transfer learning methods (Hehe et al. 2018; Peng et al. 2016; Deng et al. 2018) are proposed upon the success of deep learning (Dong et al. 2018a; Li, Tang, and Mei 2018).”
3FabRec: Fast Few-Shot Face Alignment by Reconstruction (2019, background): “This paradigm makes our approach inherently more robust against overfitting to specific training datasets as well as against human annotation variability [14].”
General Facial Representation Learning in a Visual-Linguistic Manner (2021, background): “NMEinter-ocular ↓ Method Common Challenge Full SAN [26] 3.”
A Survey on Generative Adversarial Networks: Variants, Applications, and Training (2020, background): “…detection [77 - 80], 3D image synthesis [81, 82], medical applications [86, 87], facial makeup transfer [94 - 96], facial landmark d 1 etection [97, 98], image super-resolution [99 - 102], texture synthesis [103, 105, 106], sketch synthesis [107, 110, 111], image-to-image translation [34, 112…”