Racial disparities in automated speech recognition

By Allison Koenecke, A. Nam, Emily Lake, Joe Nudell, Minnie Quartey, Zion Mengesha, Connor Toups, J. Rickford, Dan Jurafsky, Sharad Goel. 2020

792 citations recorded by the source corpus.

Significance Automated speech recognition (ASR) systems are now used in a variety of applications to convert spoken language to text, from virtual assistants, to closed captioning, to hands-free computing. By analyzing a large corpus of sociolinguistic interviews with white and African American speakers, we demonstrate large racial disparities in the performance of five popular commercial ASR systems. Our results point to hurdles faced by African Americans in using increasingly widespread tools driven by speech recognition technology. More generally, our work illustrates the need to audit emerging machine-learning systems to ensure they are broadly inclusive. Automated speech recognition (ASR) systems, which use sophisticated machine-learning algorithms to convert spoken language to text, have become increasingly widespread, powering popular virtual assistants, facilitating automated closed captioning, and enabling digital dictation platforms for health care. Over the last several years, the quality of these systems has dramatically improved, due both to advances in deep learning and to the collection of large-scale datasets used to train the systems. There is concern, however, that these tools do not work equally well for all subgroups of the population. Here, we examine the ability of five state-of-the-art ASR systems—developed by Amazon, Apple, Google, IBM, and Microsoft—to transcribe structured interviews conducted with 42 white speakers and 73 black speakers. In total, this corpus spans five US cities and consists of 19.8 h of audio matched on the age and gender of the speaker. We found that all five ASR systems exhibited substantial racial disparities, with an average word error rate (WER) of 0.35 for black speakers compared with 0.19 for white speakers. We trace these disparities to the underlying acoustic models used by the ASR systems as the race gap was equally large on a subset of identical phrases spoken by black and white individuals in our corpus. We conclude by proposing strategies—such as using more diverse training datasets that include African American Vernacular English—to reduce these performance differences and ensure speech recognition technology is inclusive.

Subfields: Artificial Intelligence, General Health Professions, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In General Health Professions

In Safety, Robustness & Interpretability

Citations in the corpus

Ethical and social risks of harm from Language Models (2021, background): “In adjacent machine learning technologies, lower performance for historically marginalised groups has often been shown, for example in facial recognition (Buolamwini and Gebru, 2018) and in speech recognition (Koenecke et al., 2020).”
Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs (2021, methodology): “[49] showed that five commercially available speech-recognition systems had much higher word error rates for Black people than for white people.”
Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “Impacts of norm and value impositions are still being determined, but can manifest in a given use case [130].”
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “Models may fail to comprehend or generate “non-standard” dialects and sociolects, essentially excluding their speakers from their user base (Joshi et al., 2020; Koenecke et al., 2020; Winata et al., 2021).”
Robust Speech Recognition via Large-Scale Weak Supervision (2022, neutral): “To check this, we compared the performance of Whisper using our normalizer versus an independently developed one from the FairSpeech project (Koenecke et al., 2020).”
Data Governance in the Age of Large-Scale Data-Driven Language Technology (2022, background): “Unsurprisingly, gendered and racial disparities have been documented in a number of language technologies [37, 79, 141], and processes of creating resources and technologies may further entrench such disparities [25, 38, 144].”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “…in applications that call for good performance across a wide range of users, including many user-facing applications such as speech recognition (Koenecke et al., 2020; Tatman, 2017) and personalized recommender systems (Patro et al., 2020), tools used for analysis of individuals such as…”
Debiasing Pre-Trained Language Models via Efficient Fine-Tuning (2022, background): “Bias Issues in Machine Learning Unfair behaviors have been found in many machine learning and artificial intelligence applications, including facial recognition (Raji and Buolamwini, 2019), recommendation systems (Schnabel et al., 2016), and speech recognition (Koenecke et al., 2020).”
FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance (2023, neutral): “Its goal is to provide the same output as a large LLM at lower latency.”
Learning to Recognize Dialect Features (2020, methodology): “Recently, a DDM was used to evaluate the performance of speech recognition systems by the density of AAVE features (Koenecke et al., 2020).”
Shifting machine learning for healthcare from development to deployment and from models to data (2022, background): “In fact, it has been shown that ML models can be biased against individuals of certain races 170 or genders 42 , or particular religious 171 or socioeconomic 15 backgrounds.”
The De-democratization of AI: Deep Learning and the Compute Divide in Artificial Intelligence Research (2020, background): “Diversity and Innovation Outcome in AI research Our results are concerning given that AI technologies have shown biases against people of color due to a lack of proper training data and oversight (Buolamwini & Gebru, 2018; Koenecke et al., 2020).”
WER we are and WER we think we are (2020, background): “Recently published datasets for spoken question answering are synthetically produced by text-tospeech systems (Lee et al., 2018; Li et al., 2018).”
Scaling Speech Technology to 1, 000+ Languages (2023, methodology): “This can lead to undesirable biases in the performance of these models [Koenecke et al., 2020].”
AI recognition of patient race in medical imaging: a modelling study (2021, neutral): “These experiments were: (1) to assess the ability of deep learning AI models to recognise race from medical images, including the ability of these models to generalise to new environments and across multiple imaging modalities; (2) to examine possible confounding anatomic and phenotype population fe”
Computational bioacoustics with deep learning: a review and roadmap (2021, background): “…(1) in most cases they must be treated as ‘black boxes’ whose detailed behaviour in response to new data is an empirical question; (2) they can carry a high risk of making biased decisions, which usually occurs because they faithfully reproduce biases in the training data (Koenecke et al., 2020).”
Sociotechnical Harms: Scoping a Taxonomy for Harm Reduction (2022, background): “Research on automatic speech recognition, for instance, has found substantial disparities in word error rates between Black and white speakers (0.35 and 0.19 respectively) [107].”
“It’s Kind of Like Code-Switching”: Black Older Adults’ Experiences with a Voice Assistant for Health Information Seeking (2021, background): “As a consequence, we know less about interactions and experiences with voice assistants among subgroups of older adults that may experience additional barriers to technology engagement related to demographic factors of identity outside of age, which have been found to be important factors of interac”