Exploring the Role of Grammar and Word Choice in Bias Toward African American English (AAE) in Hate Speech Classification
By Camille Harris, Matan Halevy, Ayanna M. Howard, Amy S. Bruckman, Diyi Yang. 2022
56 citations recorded by the source corpus.
Language usage on social media varies widely even within the context of American English. Despite this, the majority of natural language processing systems are trained only on “Standard American English,” or SAE, the construction of English most prominent among white Americans. For hate speech classification, prior work has shown that African American English (AAE) is more likely to be misclassified as hate speech. This has harmful implications for Black social media users as it reinforces and exacerbates existing notions of anti-Black racism. While past work has highlighted the relationship between AAE and hate speech classification, no work has explored the linguistic characteristics of AAE that lead to misclassification. Our work uses Twitter datasets for AAE dialect and hate speech classifiers to explore the fine-grained relationship between specific characteristics of AAE such as word choice and grammatical features and hate speech predictions. We further investigate these biases by removing profanity and examining the influence of four aspects of AAE grammar that are distinct from SAE. Results show that removing profanity accounts for a roughly 20 to 30% reduction in the percentage of samples classified as ’hate’ ’abusive’ or ’offensive,’ and that similar classification patterns are observed regardless of grammar categories.
Subfields: Artificial Intelligence, Communication, Sentiment & Opinion Analysis, AI Ethics, Fairness & Society
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yulia Tsvetkov
- Shangbin Feng
- Yuhan Liu
- Chan Young Park
- Sachin Kumar
- Vidhisha Balachandran
- Mohit Bansal
- Noah A. Smith
In Communication
In Sentiment & Opinion Analysis
In AI Ethics, Fairness & Society
Citations in the corpus
- Prompting GPT-3 To Be Reliable (2022, unknown): “…et al., 2018; Zhao et al., 2018) and other categories of social biases (Nangia et al., 2020; Nadeem et al., 2021; Parrish et al., 2022), perform poorly against minority demographic groups (Koh et al., 2021; Harris et al., 2022) or dialectical variations (Ziems et al., 2022; Tan et al., 2020).”
- From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models (2023, background): “, 2022) and data bias (Park et al., 2018; Dixon et al., 2018; Dodge et al., 2021; Harris et al., 2022) are the cause of this impact, and some studies have investigated the connection between training data and downstream task model behavior (Gonen and Webster, 2020; Li et al.”