Testifying while black: An experimental study of court reporter accuracy in transcription of African American English

By Taylor Jones, Jessica Rose Kalbfeld, Ryan Hancock, Robin Clark. 2019

Court reporters are certified at either 95% or 98% accuracy, depending on their certifying organization; however, the measure of accuracy is not one that evaluates their ability to transcribe nonstandard dialects. Here, we demonstrate that Philadelphia court reporters consistently fail to meet this level of transcription accuracy when confronted with mundane examples of spoken African American English (AAE). Furthermore, we show that they often cannot demonstrate understanding of what is being said. We show that the different morphosyntax of AAE, the different phonological patterns of AAE, and the different accents in Philadelphia related to residential segregation all conspire to produce transcriptions that not only are inaccurate, but also change the official record of who performed what actions under which circumstances, with potentially dramatic legal repercussions for everyday speakers of AAE.*

Subfields: Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “, 2010), when seeking housing (Baugh, 2018), and in the judicial system, where their testimony is misunderstood or, worse yet, disbelieved (Rickford and King, 2016; Jones et al., 2019).”
Dialect Diversity in Text Summarization on Twitter (2020, background): “While the AAE dialect is not necessarily only used by African-Americans, it is primarily associated with them and studies have shown disparate treatment of AAE dialect can lead to racial bias [62, 34].”