Stereotypical Bias Removal for Hate Speech Detection Task using Knowledge-based Generalizations

By Pinkesh Badjatiya, Manish Gupta, Vasudeva Varma. 2019

111 citations recorded by the source corpus.

With the ever-increasing cases of hate spread on social media platforms, it is critical to design abuse detection mechanisms to pro-actively avoid and control such incidents. While there exist methods for hate speech detection, they stereotype words and hence suffer from inherently biased training. Bias removal has been traditionally studied for structured datasets, but we aim at bias mitigation from unstructured text data. In this paper, we make two important contributions. First, we systematically design methods to quantify the bias for any model and propose algorithms for identifying the set of words which the model stereotypes. Second, we propose novel methods leveraging knowledge-based generalizations for bias-free learning. Knowledge-based generalization provides an effective way to encode knowledge because the abstraction they provide not only generalizes content but also facilitates retraction of information from the hate speech detection classifier, thereby reducing the imbalance. We experiment with multiple knowledge generalization policies and analyze their effect on general performance and in mitigating bias. Our experiments with two real-world datasets, a Wikipedia Talk Pages dataset (WikiDetox) of size ~ 96k and a Twitter dataset of size ~ 24k, show that the use of knowledge-based generalizations results in better performance by forcing the classifier to learn from generalized content. Our methods utilize existing knowledge-bases and can easily be extended to other tasks.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “…and O’Connor (2017); Blodgett et al. (2018); Curry and Rieser (2018); Dixon et al. (2018); Park et al. (2018); Thelwall (2018); Webster et al. (2018); Badjatiya et al. (2019); Bagdasaryan et al. (2019); Bamman et al. (2019); Bhargava and Forsyth (2019); Cao and Daumé (2019); Font and Costa-jussà…”
Challenges and frontiers in abusive content detection (2019, background): “Recent research has shown that systems often perform better for content aimed against certain targets, such as women rather than men (Badjatiya, Gupta, & Varma, 2019; Ji Ho Park, Shin, & Fung, 2018).”
Cyberbullying Detection With Fairness Constraints (2020, background): “detect biassensitive words and replace them with neutral words or tokens to reduce stereotypical bias for hate speech detection.(10) Nozza et al.”
Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics (2021, background): “In yet another line of research, Badjatiya et al. (2019) detect bias through identifying bias sensitive words.”
Handling Bias in Toxic Speech Detection: A Survey (2022, methodology): “[4] did not account for the newly acquired biases in the dataset post-mitigation.”
FuzzE: Fuzzy Fairness Evaluation of Offensive Language Classifiers on African-American English (2020, methodology): “, 2018, 2019), text classification models (Garg et al., 2018; Dixon et al., 2018; Park et al., 2018; Badjatiya et al., 2019), and in machine translation systems (Font and Costa-jussà, 2019; Escudé Font, 2019).”