Learning Deep Representation for Face Alignment with Auxiliary Attributes

By Zhanpeng Zhang, Ping Luo, Chen Change Loy, Xiaoou Tang. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014

433 citations recorded by the source corpus.

In this study, we show that landmark detection or face alignment task is not a single and independent problem. Instead, its robustness can be greatly improved with auxiliary information. Specifically, we jointly optimize landmark detection together with the recognition of heterogeneous but subtly correlated facial attributes, such as gender, expression, and appearance attributes. This is non-trivial since different attribute inference tasks have different learning difficulties and convergence rates. To address this problem, we formulate a novel tasks-constrained deep model, which not only learns the inter-task correlation but also employs dynamic task coefficients to facilitate the optimization convergence when learning multiple complex tasks. Extensive evaluations show that the proposed task-constrained learning (i) outperforms existing face alignment methods, especially in dealing with faces with severe occlusion and pose variation, and (ii) reduces model complexity drastically compared to the state-of-the-art methods based on cascaded deep model.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/ac039ca73684241f747645dd72be8f8b03edd191

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Unsupervised object learning from dense equivariant image labelling (2017, methodology): “We use the standard test split for evaluation of the MAFL dataset [33], containing 1000 images.”
Fully-Adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification (2016, background): “It has also been shown that facial attribute prediction is helpful as an auxiliary task for improving face detection [47] and face alignment [50].”
Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression (2019, neutral): “…CFAN ECCV 14 [68] 5.50 16.78 7.69 SDM CVPR 13 [63] 5.57 15.40 7.50 LBF CVPR 14 [45] 4.95 11.98 6.32 CFSS CVPR 15 [71] 4.73 9.98 5.76 TCDCN 16’ [70] 4.80 8.60 5.54 MDM CVPR 16 [54] 4.83 10.14 5.88 RAR ECCV 16 [62] 4.12 8.35 4.94 DVLN CVPR 17 [59] 3.94 7.62 4.66 TSR CVPR 17 [37] 4.36 7.56 4.99…”
Unsupervised Learning of Object Landmarks through Conditional Image Generation (2018, methodology): “On both MAFL and AFLW datasets, at 2 .”
WIDER FACE: A Face Detection Benchmark (2015, background): “Introduction Face detection is a critical step to all facial analysis algorithms, including face alignment [38, 24, 30], face recognition [27], face verification [26, 25], and face parsing [20].”
Unsupervised Discovery of Object Landmarks as Structural Representations (2018, methodology): “For human faces, there are active appearance models [10, 38, 11], template-based methods [42, 83], regression-based meth-ods [61, 12, 7, 48], and more recent methods based on deep neural networks [55, 77, 81, 82, 75, 70, 68, 33, 71].”
Deep Dual Learning for Semantic Image Segmentation (2017, methodology): “In the first category, methods [13, 23, 28] employed multitask learning with missing labels, where the labelmap L is missing and treated as an unobserved variable.”
Multi-Task Convolutional Neural Network for Pose-Invariant Face Recognition (2017, methodology): “For CNN-based MTL, previous work [64] has found that CNN learns shared features for facial landmark localization and attribute classifications, e.”
From Facial Expression Recognition to Interpersonal Relation Prediction (2016, background): “lem Formulation and Overview A natural way to learn a deep representation that captures multiple attributes is by training a multitask network that jointly predict these attributes given a face image [84]. This can be implemented directly by introducing multiple supervisory tasks during the network ”
HyperFace: A Deep Multi-Task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition (2019, neutral): “59 TCDCN [66] 8.”
Geolocation Estimation of Photos Using a Hierarchical Model and Scene Classification (2018, background): “Adding another (complementary) task has proven to be efficient to improve the results of the main task [7,17,32,47].”
Repurposing GANs for One-Shot Semantic Part Segmentation (2021, methodology): “For reference, our method using 10 labels is only slightly worse than a fully supervised baseline TCDCN [69] (8.28 versus 7.93) that uses 10,000 labels.”
Heterogeneous Face Attribute Estimation: A Deep Multi-Task Learning Approach (2017, methodology): “Following the success of MTL in neural networks [20], [47], [48], we choose to use Deep Multi-task Learning (DMTL) for estimating multiple attributes from a single face image.”
Whole-Body Human Pose Estimation in the Wild (2020, neutral): “For facial keypoint localization, cascaded networks [6,55,58,66] and multi-task learning [57,68] are widely adopted.”
Unsupervised Learning of Object Landmarks by Factorized Spatial Embeddings (2017, methodology): “We follow the standard evaluation procedure in [74] and report errors in inter-ocular distance (IOD) in table 1.”
Facial Landmark Detection: A Literature Survey (2018, methodology): “Diﬀerent from [119] [120], it combines features from multiple convolutional layers to leverage both the coarse and ﬁne feature representations.”