Learning Race from Face: A Survey

By Si-Yao Fu, Haibo He, Zeng-Guang Hou. 2014

174 citations recorded by the source corpus.

Faces convey a wealth of social signals, including race, expression, identity, age and gender, all of which have attracted increasing attention from multi-disciplinary research, such as psychology, neuroscience, computer science, to name a few. Gleaned from recent advances in computer vision, computer graphics, and machine learning, computational intelligence based racial face analysis has been particularly popular due to its significant potential and broader impacts in extensive real-world applications, such as security and defense, surveillance, human computer interface (HCI), biometric-based identification, among others. These studies raise an important question: How implicit, non-declarative racial category can be conceptually modeled and quantitatively inferred from the face? Nevertheless, race classification is challenging due to its ambiguity and complexity depending on context and criteria. To address this challenge, recently, significant efforts have been reported toward race detection and categorization in the community. This survey provides a comprehensive and critical review of the state-of-the-art advances in face-race perception, principles, algorithms, and applications. We first discuss race perception problem formulation and motivation, while highlighting the conceptual potentials of racial face processing. Next, taxonomy of feature representational models, algorithms, performance and racial databases are presented with systematic discussions within the unified learning scenario. Finally, in order to stimulate future research in this field, we also highlight the major opportunities and challenges, as well as potentially important cross-cutting themes and research directions for the issue of learning race from face.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

Citations in the corpus

Diversity in Faces (2019, background): “As prior work has pointed out, skin color alone is not a strong predictor of race, and other features such as facial proportions are important [6,63–65].”
Face Recognition: Too Bias, or Not Too Bias? (2020, methodology): “This was then further reﬁned by processing faces using ethnicity [12] and gender [20] classiﬁers.”
Deciphering the 2016 U.S. Presidential Campaign in the Twitter Sphere: A Comparison of the Trumpists and Clintonists (2016, background): “(Fu, He, and Hou 2014) provides a comprehensive survey of race classification based on facial features.”
Truth from the machine: artificial intelligence and the materialization of identity (2021, background): “The same is true of other e ﬀ orts to infer identity from the face, in domains from disability and gender to race (Hashemi et al. 2012; Bautista et al. 2015; Fu, He, and Hou 2014).”
Understanding Unequal Gender Classification Accuracy from Face Images (2018, background): “(Note that gender classiﬁcation is a distinct task from race classiﬁcation [15].)”
Can you tell where in India I am from? Comparing humans and computers on fine-grained race face classification (2017, methodology): “Coarse race classiﬁcation be-tween groups like caucasians and black have been explored in psychology [1][4] and interestingly it has been shown that humans can reliably perform this task in the absence of salient cues such as skin colour, expressions, cosmetics, ornaments or attributes such as hair…”
InclusiveFaceNet: Improving Face Attribute Detection with Race and Gender Diversity (2017, background): “At the core of this work lies the idea that faces look different across different races and genders (S et al., 2014).”
What Else Does Your Biometric Data Reveal? A Survey on Soft Biometrics (2016, background): “[55] which provides a comprehensive study on human and automated analysis of race classification.”
How We've Taught Algorithms to See Identity: Constructing Race and Gender in Image Databases for Facial Analysis (2020, background): “Like PPB, these different sources also drove the annotation process: As prior work has pointed out, skin color alone is not a strong predictor of race, and other features such as facial proportions are important [50, 54, 130, 131].”
Deep Learning for Face Recognition: Pride or Prejudiced? (2019, background): “The own-race bias in humans has been well established, where we are able to recognize people of our own race more easily as compared to individuals of other race [26, 14].”
Improving Smiling Detection with Race and Gender Diversity (2017, background): “d then transferring that learning into a new model. 4. Learning Diversity 4.1. Demographic Accuracy At the core of this work lies the idea that faces look different across different races and genders [34, 6], and that it is equally important to do well on each demographic group. To the best of our k”