Intersectional Bias in Hate Speech and Abusive Language Datasets

By Jae-Yeon Kim, Carlos Ortiz, Sarah Nam, Sarah Santiago, Vivek Datta. 2020

50 citations recorded by the source corpus.

Algorithms are widely applied to detect hate speech and abusive language in social media. We investigated whether the human-annotated data used to train these algorithms are biased. We utilized a publicly available annotated Twitter dataset (Founta et al. 2018) and classified the racial, gender, and party identification dimensions of 99,996 tweets. The results showed that African American tweets were up to 3.7 times more likely to be labeled as abusive, and African American male tweets were up to 77% more likely to be labeled as hateful compared to the others. These patterns were statistically significant and robust even when party identification was added as a control variable. This study provides the first systematic evidence on intersectional bias in datasets of hate speech and abusive language.

Subfields: Artificial Intelligence, Communication, Social Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yejin Choi
2. Noah A. Smith
3. Mohit Bansal
4. Maarten Sap
5. Swabha Swayamdipta
6. Percy Liang
7. Jianfeng Gao
8. Luke S. Zettlemoyer

In Communication

1. William S. Isaac
2. Tanmoy Chakraborty
3. Xuhui Zhou
4. Noah A. Smith
5. Yejin Choi
6. Tanmay Garg
7. Tharun Suresh
8. Swabha Swayamdipta

In Social Psychology

1. Yejin Choi
2. Iason Gabriel
3. Noah A. Smith
4. Geoffrey Irving
5. Maarten Sap
6. Zachary Kenton
7. Laura Weidinger
8. Abeba Birhane

Citations in the corpus

• Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, neutral): “…(2019); Vaidya et al. (2019); Webster et al. (2019); Ethayarajh (2020); Gaut et al. (2020); Gencoglu (2020); Hovy et al. (2020); Huang et al. (2020); Kim et al. (2020); Peng et al. (2020); Ravfogel et al. (2020); Rios (2020); Sap et al. (2020); Saunders and Byrne (2020); Sheng et al. (2020);…”
• Challenges in Automated Debiasing for Toxic Language Detection (2021, background): “Current hate speech or toxic language detection1 systems exhibit problematic and discriminatory behavior that causes them to have disparate negative impact on minority populations (Yasin, 2018; Guynn, 2020; Kim et al., 2020; Dias Oliva et al., 2020).”
• Taxonomy of Risks posed by Language Models (2022, background): “showing that more work is needed to address discrimination and hate speech risks in tandem [49, 103, 165, 194].”
• Theories of “Gender” in NLP Bias Research (2022, background): “[76] Jae Yeon Kim, Carlos Ortiz, Sarah Nam, Sarah Santiago, and Vivek Datta.”
• Handling Bias in Toxic Speech Detection: A Survey (2022, background): “[51] study the intersection of racial and gender bias for AAE.”