Demographic Dialectal Variation in Social Media: A Case Study of African-American English

By Su Lin Blodgett, Lisa Green, Brendan T. O'Connor. 2016

406 citations recorded by the source corpus.

Though dialectal language is increasingly abundant on social media, few resources exist for developing NLP tools to handle such language. We conduct a case study of dialectal language in online conversational text by investigating African-American English (AAE) on Twitter. We propose a distantly supervised model to identify AAE-like language from demographics associated with geo-located messages, and we verify that this language follows well-known AAE linguistic phenomena. In addition, we analyze the quality of existing language identification and dependency parsing tools on AAE-like text, demonstrating that they perform poorly on such text compared to text associated with white speakers. We also provide an ensemble classifier for language identification which eliminates this disparity and release a new corpus of tweets containing AAE-like language.

Subfields: Artificial Intelligence, Language and Linguistics, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Language and Linguistics

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “Other representational harms Hovy and Søgaard (2015); Blodgett et al. (2016); Bolukbasi et al. (2016b); Hovy and Spruit (2016); Blodgett and O’Connor (2017); Larson (2017); Schnoebelen (2017); Blodgett et al. (2018); Curry and Rieser (2018); Díaz et al. (2018); Dixon et al. (2018); Kiritchenko and…”
Challenges in Detoxifying Language Models (2021, methodology): “To study the social bias amplification, we use the BOLD dataset (Dhamala et al., 2021) and TWITTERAAE dataset (Blodgett et al., 2016).”
Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (2022, background): “…using large web-scraped datasets, and to Luccioni and Viviano (2021) and Dodge et al. (2021) for analyses of such large-scale web-scraped corpora and their representational, legal, consent, and PII issues. text with AAVE features (Jørgensen et al., 2015, 2016; Blodgett et al., 2016, among others).”
Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus (2021, methodology): “We use a dialect-aware topic model from Blodgett et al. (2016), which was trained on 60M geolocated tweets and relies on US census race/ethnicity data as topics.”
The Risk of Racial Bias in Hate Speech Detection (2019, methodology): “In the dialect priming condition, we explicitly include the tweet’s dialect as measured by Blodgett et al. (2016), as well as extra instructions priming workers to think of tweet dialect as a proxy for the author’s race.”
Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems (2018, background): “This paper does not examine the differences in accuracies of systems on text produced by different races or genders, as was done by Hovy (2015); Blodgett et al. (2016); Jurgens et al. (2017); Buo-lamwini and Gebru (2018).”
Social Bias Frames: Reasoning about Social and Power Implications of Language (2019, background): “We fnd that our dataset is predominantly written in White-aligned English (78% of posts), as measured by a lexical dialect detector by Blodgett et al. (2016), with <10% of posts having indicators of African-American English.”
Racial Bias in Hate Speech and Abusive Language Detection Datasets (2019, methodology): “We ﬁnd evidence of systematic racial biases across all of the classiﬁers, with AAE tweets predicted as belonging to negative classes like hate speech or harassment signiﬁcantly more frequently than SAE tweets.”
Differential Privacy Has Disparate Impact on Model Accuracy (2019, background): “This task involves classifying Twitter posts from the recently proposed corpus of African-American English [4, 5] as positive or negative.”
Fairness Without Demographics in Repeated Loss Minimization (2018, background): “Jurgens et al., 2017), dependency parsing (Blodgett et al., 2016), part-of-speech tagging (Hovy & Sgaard, 2015), academic recommender systems (Sapiezynski et al., 2017), and automatic video captioning (Tatman, 2017).”
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019, background): “Such variations were also observed across racial or ethnic groups (Blodgett et al., 2016).”
Adversarial Removal of Demographic Attributes from Text Data (2018, background): “…race, gender, and many other social factors of the author (Kop-pel et al., 2002; Burger et al., 2011; Nguyen et al., 2013; Weren et al., 2014; Verhoeven and Daele-mans, 2014; Rangel et al., 2016; Verhoeven et al., 2016; Blodgett et al., 2016), or even the audience of the text (Voigt et al., 2018).”
Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection (2020, methodology): “We follow the setup of Elazar and Goldberg (2018) which used a twitter dataset, collected by Blodgett et al. (2016), where each tweet is associated with “race” information and a sentiment which was determined by their belonging to some emoji group.”
Challenges in Automated Debiasing for Toxic Language Detection (2021, methodology): “In our experiments, we estimate the dialect of a tweet using a topic model from Blodgett et al. (2016).”
Dats Wassup!!: Investigating African-American Vernacular English in Transformer-Based Text Generation (2020, methodology): “Sample Identification TwitterAAE (Blodgett et al., 2016) collects AAVE tweets by using a distantly supervised mixed-membership model on samples that are geolocated to African-American blockgroups, as defined by the U.”
Style Transfer Through Back-Translation (2018, background): “Moreover, prior work has shown that the quality of language identification and POS tagging degrades significantly on African American Vernacular English (Blodgett et al., 2016; Jørgensen et al., 2015); YouTube’s automatic captions have higher error rates for women and speakers from Scotland (Rudinge”