Diagnosing Gender Bias in Image Recognition Systems

By Carsten Schwemmer, C. Knight, Emily Bello-Pardo, Stan Oklobdzija, Martijn Schoonvelde, Jeffrey W. Lockhart. 2020

62 citations recorded by the source corpus.

Image recognition systems offer the promise to learn from images at scale without requiring expert knowledge. However, past research suggests that machine learning systems often produce biased output. In this article, we evaluate potential gender biases of commercial image recognition platforms using photographs of U.S. members of Congress and a large number of Twitter images posted by these politicians. Our crowdsourced validation shows that commercial image recognition systems can produce labels that are correct and biased at the same time as they selectively report a subset of many possible true labels. We find that images of women received three times more annotations related to physical appearance. Moreover, women in images are recognized at substantially lower rates in comparison with men. We discuss how encoded biases such as these affect the visibility of women, reinforce harmful gender stereotypes, and limit the validity of the insights that can be gathered from such data.

Subfields: Computer Vision and Pattern Recognition, Experimental and Cognitive Psychology, Gender Studies, Sociology and Political Science, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Experimental and Cognitive Psychology

In Sociology and Political Science

In Safety, Robustness & Interpretability

Citations in the corpus

Learning Transferable Visual Models From Natural Language Supervision (2021, methodology): “…given the combined set of labels that Google Cloud Vision (GCV), Amazon Rekognition and Microsoft returned for all the images, similar to the biases Schwemmer et al. (2020) found in GCV systems, we found our system also disproportionately attached labels to do with hair and appearance in general…”
Flamingo: a Visual Language Model for Few-Shot Learning (2022, background): “[97] Carsten Schwemmer, Carly Knight, Emily D.”
Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision (2022, background): “These harms can be bigger if the type of predicted labels are derogatory or reinforce harmful stereotypes [12, 109].”