Improving Smiling Detection with Race and Gender Diversity

By Hee Jung Ryu, Margaret Mitchell, Hartwig Adam. 2017

32 citations recorded by the source corpus.

Recent progress in deep learning has been accompanied by a growing concern for whether models are fair for users, with equally good performance across different demographics. In computer vision research, such questions are relevant to face detection and the related task of face attribute detection, among others. We measure race and gender inclusion in the context of smiling detection, and introduce a method for improving smiling detection across demographic groups. Our method introduces several modifications over existing detection methods, leveraging twofold transfer learning to better model facial diversity. Results show that this technique improves accuracy against strong baselines for most demographic groups as well as overall. Our best-performing model defines a new state-of-the-art for smiling detection, reaching 91% on the Faces of the World dataset. The accompanying multi-head diversity classifier also defines a new state-of-the-art for gender classification, reaching 93.87% on the Faces of the World dataset. This research demonstrates the utility of modeling race and gender to improve a face attribute detection task, using a twofold transfer learning framework that allows for privacy towards individuals in a target dataset.

Subfields: Computer Vision and Pattern Recognition, Experimental and Cognitive Psychology, Law, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Experimental and Cognitive Psychology

In Law

In AI Ethics, Fairness & Society

Citations in the corpus

Women also Snowboard: Overcoming Bias in Captioning Models (2018, background): “, gender, ethnic biases) have been studied across a wide variety of AI domains [5,6,7,8,9,10].”
Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems (2018, background): “Approaches on how to mitigate inappropriate biases (Schmidt, 2015; Bolukbasi et al., 2016; Kilbertus et al., 2017; Ryu et al., 2017; Speer, 2017; Zhang et al., 2018; Zhao et al., 2018) are also beyond the scope of this paper.”
Adversarial Removal of Gender from Deep Image Representations (2018, unknown): “More recently, there has been work addressing different types of biases in image data [22, 29, 23, 4, 18, 5].”
Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation (2019, methodology): “Finally, we veriﬁed our ﬁndings on the real-world CelebA dataset [32], used in [41] to study face attribute recognition when the presence of attributes, e.g., “smiling,” is correlated with gender.”
Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations (2018, background): “There are also works addressing biases in images [25, 39, 27, 3, 20, 4].”