Empirical Analysis of Multi-Task Learning for Reducing Model Bias in Toxic Comment Detection

By Ameya Vaidya, Feng Mai, Yue Ning. 2019

23 citations recorded by the source corpus.

With the recent rise of toxicity in online conversations on social media platforms, using modern machine learning algorithms for toxic comment detection has become a central focus of many online applications. Researchers and companies have developed a variety of shallow and deep learning models to identify toxicity in online conversations, reviews, or comments with mixed successes. However, these existing approaches have learned to incorrectly associate non-toxic comments that have certain trigger-words (e.g. gay, lesbian, black, muslim) as a potential source of toxicity. In this paper, we evaluate dozens of state-of-the-art models with the specific focus of reducing model bias towards these commonly-attacked identity groups. We propose a multi-task learning model with an attention layer that jointly learns to predict the toxicity of a comment as well as the identities present in the comments in order to reduce this bias. We then compare our model to an array of shallow and deep-learning models using metrics designed especially to test for unintended model bias within these identity groups.

Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yejin Choi
2. Yulia Tsvetkov
3. Noah A. Smith
4. Maarten Sap
5. Swabha Swayamdipta
6. Mohit Bansal
7. Percy Liang
8. Luke S. Zettlemoyer

In Molecular Biology

1. Yejin Choi
2. Noah A. Smith
3. Yulia Tsvetkov
4. Hal Daumé Iii
5. Hanna M. Wallach
6. Oguzhan Gencoglu
7. Swabha Swayamdipta
8. Maarten Sap

In Natural Language Processing

1. Yulia Tsvetkov
2. Yejin Choi
3. Noah A. Smith
4. Maarten Sap
5. Swabha Swayamdipta
6. Anjalie Field
7. Mohit Bansal
8. Hal Daumé Iii

In Safety, Robustness & Interpretability

1. Yulia Tsvetkov
2. Yejin Choi
3. Maarten Sap
4. Noah A. Smith
5. Swabha Swayamdipta
6. Anjalie Field
7. Su Lin Blodgett
8. Xuhui Zhou

Citations in the corpus

• Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “…Prabhakaran et al. (2019); Romanov et al. (2019); Sap et al. (2019); Sedoc and Ungar (2019); Stanovsky et al. (2019); Sweeney and Najafian (2019); Vaidya et al. (2019); Zhiltsova et al. (2019); Chopra et al. (2020); Gonen and Webster (2020); Gyamfi et al. (2020); Hube et al. (2020); Ravfogel et…”
• Challenges in Automated Debiasing for Toxic Language Detection (2021, background): “, 2018), or adversarial feature learning (Vaidya et al., 2019).”
• Demoting Racial Bias in Hate Speech Detection (2020, background): “Vaidya et al. (2019) show that multitask training on a related task e.g., identity prediction, allows the model to shift focus to toxic-related elements in hate speech detection.”
• Cyberbullying Detection With Fairness Constraints (2020, methodology): “…model to retrieve informative numerical representations of the textual data (e.g. representations of tokens, words, sentences, or paragraphs) and training a deep neural network classifier with these representations [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25].”
• Theories of “Gender” in NLP Bias Research (2022, background): “8 [148] Ameya Vaidya, Feng Mai, and Yue Ning.”