Handling Bias in Toxic Speech Detection: A Survey

By Tanmay Garg, Sarah Masud, Tharun Suresh, Tanmoy Chakraborty. 2022

133 citations recorded by the source corpus.

Detecting online toxicity has always been a challenge due to its inherent subjectivity. Factors such as the context, geography, socio-political climate, and background of the producers and consumers of the posts play a crucial role in determining if the content can be flagged as toxic. Adoption of automated toxicity detection models in production can thus lead to a sidelining of the various groups they aim to help in the first place. It has piqued researchers’ interest in examining unintended biases and their mitigation. Due to the nascent and multi-faceted nature of the work, complete literature is chaotic in its terminologies, techniques, and findings. In this article, we put together a systematic study of the limitations and challenges of existing methods for mitigating bias in toxicity detection. We look closely at proposed methods for evaluating and mitigating bias in toxic speech detection. To examine the limitations of existing methods, we also conduct a case study to introduce the concept of bias shift due to knowledge-based bias mitigation. The survey concludes with an overview of the critical challenges, research gaps, and future directions. While reducing toxicity on online platforms continues to be an active area of research, a systematic study of various biases and their mitigation strategies will help the research community produce robust and fair models.1

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Scaling Instruction-Finetuned Language Models (2022, background): “Automated measures of toxic language contain several forms of noise and bias (Xu et al., 2021; Garg et al., 2022), and do not consider diverse perspectives on toxic language (Goyal et al.”
PaLM 2 Technical Report (2023, background): “Automated measures of toxic language contain noise and bias (Xu et al., 2021a; Garg et al., 2022), do not consider diverse perspectives (Goyal et al.”