Deep Neural Networks Are More Accurate Than Humans at Detecting Sexual Orientation From Facial Images

By Yilun Wang, Michal Kosinski. 2018

561 citations recorded by the source corpus.

We show that faces contain much more information about sexual orientation than can be perceived or interpreted by the human brain. We used deep neural networks to extract features from 35,326 facial images. These features were entered into a logistic regression aimed at classifying sexual orientation. Given a single facial image, a classifier could correctly distinguish between gay and heterosexual men in 81% of cases, and in 71% of cases for women. Human judges achieved much lower accuracy: 61% for men and 54% for women. The accuracy of the algorithm increased to 91% and 83%, respectively, given five facial images per person. Facial features employed by the classifier included both fixed (e.g., nose shape) and transient facial features (e.g., grooming style). Consistent with the prenatal hormone theory of sexual orientation, gay men and women tended to have gender-atypical facial morphology, expression, and grooming styles. Prediction models aimed at gender alone allowed for detecting gay males with 57% accuracy and gay females with 58% accuracy. Those findings advance our understanding of the origins of sexual orientation and the limits of human perception. Additionally, given that companies and governments are increasingly using computer vision algorithms to detect people’s intimate traits, our findings expose a threat to the privacy and safety of gay men and women.

Subfields: Computer Vision and Pattern Recognition, Experimental and Cognitive Psychology, Gender Studies, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Experimental and Cognitive Psychology

In Gender Studies

In Safety, Robustness & Interpretability

Citations in the corpus

Ethical and social risks of harm from Language Models (2021, background): “Tools that attempt to infer unobservable characteristics - such as sexual orientation from a portrait (Wang and Kosinski, 2018) - are inherently prone to error.”
Re-imagining Algorithmic Fairness in India and Beyond (2021, background): “, [187, 212]) potentially being exported to India and harming minorities.”
Towards fairer datasets: filtering and balancing the distribution of the people subtree in the ImageNet hierarchy (2019, background): “Examples range from quantifying beauty [47] to predicting sexual orientation [76] to (arguably) annotating gender [25, 42].”
Assessing gender bias in machine translation: a case study with Google Translate (2018, background): “ectly revived heated debates about the controversial eld of physiognomy, with proposals of AI systems capable of identifying the sexual orientation of an individual through its facial characteristics [38]. Similar concerns are growing at an unprecedented rate in the media, with reports of Apple’s Ip”
Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communities (2021, background): “In 2017, Stanford researchers attempted to build an AI ‘gaydar’, a computer vision model capable of guessing a person’s sexual orientation from images (Wang and Kosinski 2018).”
Taxonomy of Risks posed by Language Models (2022, background): “Such systems may never be accurate - for example, image classification models that attempt to infer unobservable characteristics, such as sexual orientation from a portrait [190], are inherently prone to error.”
BERTIN: Efficient Pre-Training of a Spanish Language Model using Perplexity Sampling (2022, background): “These applications, some of them modern versions of phrenology (Wang and Kosinski, 2018), have a dramatic impact on the lives of people all over the world.”
Item Response Theory—A Statistical Framework for Educational and Psychological Measurement (2021, unknown): “…researchers tend to study psychological constructs using not only item response data but also a much wider range of daily behavioral data including those from social media, facial expression, and writing and speech (e.g., Inkster et al., 2016; Nave et al., 2018; Wang and Kosinski, 2018).”
Truth from the machine: artificial intelligence and the materialization of identity (2021, background): “’ (Wang and Kosinski 2018, our emphasis).”
Pitfalls in Machine Learning Research: Reexamining the Development Cycle (2020, unknown): “Recent examples of this include algorithms for determining criminal tendencies [53, 22] and detecting sexual orientation [51] from photos of faces, as well as commercial services that automatically determine suitable job candidates for hiring managers [12].”
How artificial intelligence will change the future of marketing (2019, background): “For example, two Stanford researchers used deep neural networks to identify people ’ s sexual orientation, merely by analyzing facial images (Wang and Kosinski 2018).”
Gaydar and the Fallacy of Decontextualized Measurement (2018, methodology): “As noted above, we are not claiming that experiments such as those of Tabak and Zayas (2012) and Wang and Kosinski (2018) are useless.”
How to Design AI for Social Good: Seven Essential Factors (2020, unknown): “For example, a recent paper used publicly available images of faces uploaded to a dating website as a way to train AI software to detect someone’s sexuality based on a small number of photos (Wang and Kosinski 2018).”
Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification (2018, background): “Controversial papers such as (Kosinski and Wang, 2017) claim to determine the sexuality of Caucasian males whose proﬁle pictures are on Facebook or dating sites.”
Mitigating Bias in Gender, Age and Ethnicity Classification: A Multi-task Convolution Neural Network Approach (2018, background): “Moreover nowadays, FA is not restricted to face recognition, but has transgressed onto emotion analysis [5], gender classification [6], as well as other facial characteristics [7–9].”