Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing

By Inioluwa Deborah Raji, Timnit Gebru, Margaret Mitchell, Joy Buolamwini, Joonseok Lee, Emily L. Denton. 2020

334 citations recorded by the source corpus.

Although essential to revealing biased performance, well intentioned attempts at algorithmic auditing can have effects that may harm the very populations these measures are meant to protect. This concern is even more salient while auditing biometric systems such as facial recognition, where the data is sensitive and the technology is often used in ethically questionable manners. We demonstrate a set of fiveethical concerns in the particular case of auditing commercial facial processing technology, highlighting additional design considerations and ethical tensions the auditor needs to be aware of so as not exacerbate or complement the harms propagated by the audited system. We go further to provide tangible illustrations of these concerns, and conclude by reflecting on what these concerns mean for the role of the algorithmic audit and the fundamental product limitations they reveal.

Subfields: Computer Vision and Pattern Recognition, Pharmacy, Safety Research, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Safety Research

In Safety, Robustness & Interpretability

Citations in the corpus

Learning Transferable Visual Models From Natural Language Supervision (2021, background): “…the FairFace benchmark dataset for gender, race and age classification of images by intersectional categories, accuracy on benchmarks offers only one approximation of algorithmic fairness, as Raji et al. (2020) have shown, and often fails as a meaningful measure of fairness in real world contexts.”
Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs (2021, background): “[68], this approach may place a greater burden on members of already disadvantaged groups by requiring their participation as data subjects—itself a form of injustice.”
Designing an Online Infrastructure for Collecting AI Data From People With Disabilities (2021, background): “…be very challenging for some PWD. Scraping online data of PWD is also problematic not only because of the long-tail nature of various disabilities but also because of the ethical considerations and the heightened need to protect the sensitive demographic attributes related to disabilities [23, 46].”
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction (2022, background): “, facial features, skin tone, or voice), such as computer vision [46, 176], natural language processing [113, 131, 173], and speech recognition systems [124, 143].”
Sociotechnical Harms: Scoping a Taxonomy for Harm Reduction (2022, background): “facial features, skin tone, or voice), such as computer vision [40, 155], natural language processing [95, 112, 153], and speech recognition systems [107, 124].”
Addressing Privacy Threats from Machine Learning (2021, background): “Nevertheless, audits have limitations; for example, audits may inadvertently "normalize tasks that are inherently harmful to certain communities" (Raji et al., 2020).”
Towards Measuring Fairness in AI: The Casual Conversations Dataset (2021, background): “[19, 20] analyze the commercial impact Gender Shades [2] and discuss ethical concerns auditing facial processing technologies.”
A Human in the Loop is Not Enough: The Need for Human-Subject Experiments in Facial Recognition (2020, background): “In previous work on evaluating the performance of facial recognition systems, researchers have used datasets of images of celebrities [8] or members of parliament [1], which can lead to discrepancies between type quality of the experimental images and the images encountered in a real-world…”
Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases (2020, neutral): “question answering [11, 43, 20, 76, 47, 52].”
Are My Deep Learning Systems Fair? An Empirical Study of Fixed-Seed Training (2021, background): “In practice, such gaps are the key reason that facial recognition systems [58, 76] have a racial bias.”