How Computers See Gender

By Morgan Klaus Scheuerman, Jacob M. Paul, Jed R. Brubaker. 2019

158 citations recorded by the source corpus.

Investigations of facial analysis (FA) technologies-such as facial detection and facial recognition-have been central to discussions about Artificial Intelligence's (AI) impact on human beings. Research on automatic gender recognition, the classification of gender by FA technologies, has raised potential concerns around issues of racial and gender bias. In this study, we augment past work with empirical data by conducting a systematic analysis of how gender classification and gender labeling in computer vision services operate when faced with gender diversity. We sought to understand how gender is concretely conceptualized and encoded into commercial facial analysis and image labeling technologies available today. We then conducted a two-phase study: (1) a system analysis of ten commercial FA and image labeling services and (2) an evaluation of five services using a custom dataset of diverse genders using self-labeled Instagram images. Our analysis highlights how gender is codified into both classifiers and data standards. We found that FA services performed consistently worse on transgender individuals and were universally unable to classify non-binary genders. In contrast, image labeling often presented multiple gendered concepts. We also found that user perceptions about gender performance and identity contradict the way gender performance is encoded into the computer vision infrastructure. We discuss our findings from three perspectives of gender identity (self-identity, gender performativity, and demographic identity) and how these perspectives interact across three layers: the classification infrastructure, the third-party applications that make use of that infrastructure, and the individuals who interact with that software. We employ Bowker and Star's concepts of "torque" and "residuality" to further discuss the social implications of gender classification. We conclude by outlining opportunities for creating more inclusive classification infrastructures and datasets, as well as with implications for policy.

Subfields: Clinical Psychology, Computer Vision and Pattern Recognition, Experimental and Cognitive Psychology, AI Ethics, Fairness & Society, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Clinical Psychology

In Computer Vision and Pattern Recognition

In Experimental and Cognitive Psychology

Emily L. Denton

In AI Ethics, Fairness & Society

In Human-Computer Interaction

Citations in the corpus

Large image datasets: A pyrrhic win for computer vision? (2020, background): “0 200 400 600 800 Class index [0-999] 0 500 1000 1500 2000”
Bringing the People Back In: Contesting Benchmark Machine Learning Datasets (2020, background): “…systems abound in ways that they have failed people of color (Noble, 2018; Benjamin, 2019), women (Bolukbasi et al., 2016), LGBTQ+ communities (Scheuerman et al., 2019), people with disabilities (Hutchinson et al., 2020; Trewin, 2018), and the working class and those in poverty (Eubanks,…”
Between Subjectivity and Imposition (2020, background): “However, previous research has shown that they may contain biases that lead to discrimination and exclusion in several domains such as credit [37], the job market [70], facial recognition systems [19, 45, 71], algorithmic filtering [4, 62], and even advertisement [1].”
“It’s Complicated”: Negotiating Accessibility and (Mis)Representation in Image Descriptions of Race, Gender, and Disability (2021, background): “These interviews lead us, along with others [10, 11, 95, 97, 120] to strongly caution the use of AI that analyzes humans and human data, and we extend this explicitly to AI which generates image descriptions of people.”