Heterogeneous Face Attribute Estimation: A Deep Multi-Task Learning Approach

By Hu Han, Anil K. Jain, F. Wang, Shiguang Shan, Xilin Chen. 2017

299 citations recorded by the source corpus.

Face attribute estimation has many potential applications in video surveillance, face retrieval, and social media. While a number of methods have been proposed for face attribute estimation, most of them did not explicitly consider the attribute correlation and heterogeneity (e.g., ordinal versus nominal and holistic versus local) during feature representation learning. In this paper, we present a Deep Multi-Task Learning (DMTL) approach to jointly estimate multiple heterogeneous attributes from a single face image. In DMTL, we tackle attribute correlation and heterogeneity with convolutional neural networks (CNNs) consisting of shared feature learning for all the attributes, and category-specific feature learning for heterogeneous attributes. We also introduce an unconstrained face database (LFW+), an extension of public-domain LFW, with heterogeneous demographic attributes (age, gender, and race) obtained via crowdsourcing. Experimental results on benchmarks with multiple face attributes (MORPH II, LFW+, CelebA, LFWA, and FotW) show that the proposed approach has superior performance compared to state of the art. Finally, evaluations on a public-domain face database (LAP) with a single attribute show that the proposed approach has excellent generalization ability.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age (2019, methodology): “…[31] Celebrity 13K Model generated predictions no IMDB-WIKI [46] IMDB, WIKI 500K no FotW [13] Flickr 25K yes CACD [10] celebrity 160K no DiF [39] Flickr 1M **Skin color prediction † CelebA [37] CelebFace [54, 55] LFW [23] 200K no LFW+ [16] LFW [23 cability of computer vision methods to these ﬁelds.”
Diversity in Faces (2019, neutral): “Many other face data sets and evaluations have produced similar improvements using deep learning methods [8,9,14,22–34].”
Transferability and Hardness of Supervised Classification Tasks (2019, background): “4% lower than the best scores reported by DMTL 2018 [21] (last column of Table 1).”
Slim-CNN: A Light-Weight CNN for Face Attribute Prediction (2019, methodology): “Attributes Liu[14] MOON [15] Hand [5] DMTL [4] AFFACT [3] PSE [22] Slim (this work) 5 O’Clock Shadow 91 94.03 94.51 95 94.87 94.85 94.75 Arched Eyebrows 79 82.26 83.42 86 84.21 83.75 83.2 Attractive 81 81.67 83.06 85 82.86 82.95 82.85 Bags Under Eyes 79 84.92 84.92 99 85.31 85.35 85.33 Bald 98 98.77”
Improving Confidence Estimates for Unfamiliar Examples (2018, methodology): “Samples are split into familiar F and unfamiliar U based on age annotations provided by Han et al. [15], with familiar ages 18-59 years and unfamiliar ages outside that range.”
UM-Adapt: Unsupervised Multi-Task Adaptation Using Adversarial Cross-Task Distillation (2019, background): “Multi-task learning [3] has been applied in computer vision literature [38] for quite a long time for a varied set of tasks in consideration [27, 8, 9, 16, 43].”
FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age for Bias Measurement and Mitigation (2021, background): “LFW+ [15] LFW (Newspapers) 15K X X X merged merged no”
MTL-NAS: Task-Agnostic Neural Architecture Search Towards General-Purpose Multi-Task Learning (2020, background): “A great success has been witnessed in various areas such as detection [15, 16, 23, 50, 54, 55], depth prediction and semantic segmentation (also surface normal prediction) [10, 65], human-related tasks [21,48,60,62,68], etc.”
PAD-Net: Multi-tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsing (2018, methodology): “Deep multi-task learning[38, 44] has been widely used in various computer vision problems, such as joint inference scene geometric and semantic [21], face attribute estimation [14], simultaneous contour detection and semantic segmentation [12].”