Demographics Should Not Be the Reason of Toxicity: Mitigating Discrimination in Text Classifications with Instance Weighting

By Guanhua Zhang, Bing Bai, Junqi Zhang, Kun Bai, Conghui Zhu, Tuo Zhao. 2020

81 citations recorded by the source corpus.

With the recent proliferation of the use of text classifications, researchers have found that there are certain unintended biases in text classification datasets. For example, texts containing some demographic identity-terms (e.g., “gay”, “black”) are more likely to be abusive in existing abusive language detection datasets. As a result, models trained with these datasets may consider sentences like “She makes me happy to be gay” as abusive simply because of the word “gay.” In this paper, we formalize the unintended biases in text classification datasets as a kind of selection bias from the non-discrimination distribution to the discrimination distribution. Based on this formalization, we further propose a model-agnostic debiasing training framework by recovering the non-discrimination distribution using instance weighting, which does not require any extra resources or annotations apart from a pre-defined set of demographic identity-terms. Experiments demonstrate that our method can effectively alleviate the impacts of the unintended biases without significantly hurting models’ generalization ability.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “…(2019); Zmigrod et al. (2019); Hube et al. (2020); Hutchinson et al. (2020); Jia et al. (2020); Papakyriakopoulos et al. (2020); Popović et al. (2020); Pryzant et al. (2020); Saunders and Byrne (2020); Sen and Ganguly (2020); Shah et al. (2020); Sweeney and Najafian (2020); Zhang et al. (2020b).”
A Survey of Race, Racism, and Anti-Racism in NLP (2021, background): “Bias Develop Model Xia et al. (2020) 2020 Workshop Abusive Language Debias Zhang et al. (2020) 2020 ACL Abusive Language Detect Bias Zhao and Chang (2020) 2020 EMNLP Ethics/Task-indep.”
On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning (2021, background): “Zhao et al. (2018) summarize a list of occupations that are prone to be stereotyped in practice, leading to coreference resolutions models and occupation prediction models having biases in performance in pro- and antistereotypical instances when trained on short bios.”
Handling Bias in Toxic Speech Detection: A Survey (2022, background): “Only a few studies have explored this phenomenon from a non-binary perspective [118].”
Theories of “Gender” in NLP Bias Research (2022, background): “305 [165] Guanhua Zhang, Bing Bai, Junqi Zhang, Kun Bai, Conghui Zhu, and Tiejun Zhao.”
On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representations (2022, unknown): “The notion of bias we focus on is group disparities in sentiment , toxicity and stereotypes , which have been extensively researched in the context of fairness (Huang et al., 2020; Zhang et al., 2020; Borkan et al., 2019; Adragna et al., 2020).”