Reducing Gender Bias in Word-Level Language Models with a Gender-Equalizing Loss Function

By Yusu Qian, Urwa Muaz, J. Hyun. 2019

105 citations recorded by the source corpus.

Gender bias exists in natural language datasets, which neural language models tend to learn, resulting in biased text generation. In this research, we propose a debiasing approach based on the loss function modification. We introduce a new term to the loss function which attempts to equalize the probabilities of male and female words in the output. Using an array of bias evaluation metrics, we provide empirical evidence that our approach successfully mitigates gender bias in language models without increasing perplexity. In comparison to existing debiasing strategies, data augmentation, and word embedding debiasing, our method performs better in several aspects, especially in reducing gender bias in occupation words. Finally, we introduce a combination of data augmentation and our approach and show that it outperforms existing strategies in all bias evaluation metrics.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Language Models are Few-Shot Learners (2020, background): “The literature on this is also extensive [QMZH19, HZJ + 19], so we offer only a few brief comments on future directions speciﬁc to large language models.”
Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, neutral): “…Maudslay et al. (2019); May et al. (2019); Mirzaev et al. (2019); Prates et al. (2019); Précenth (2019); Prost et al. (2019); Pujari et al. (2019); Qian et al. (2019); Sedoc and Ungar (2019); Stanovsky et al. (2019); Tan and Celis (2019); Zhao et al. (2019); Zhou et al. (2019); Chopra et al.…”
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation (2019, background): “Employing specific templates for model evaluation is a commonly used practice (Zhao et al., 2018; Qian et al., 2019; Sheng et al., 2019), but we acknowledge that they can lack context-sensitivity, and that such evaluation is necessarily limited and not comprehensive.”
LaMDA: Language Models for Dialog Applications (2022, background): “, [74, 75, 76, 77, 78, 79, 80, 81, 82]), meaningfully addressing these issues remains an active research area.”
Societal Biases in Language Generation: Progress and Challenges (2021, background): “…2021), GPT-3 (Brown et al., 2020), CTRL (Dhamala et al., 2021), TransformerXL (Shwartz et al., 2020; Vig et al., 2020; Huang et al., 2020), and XLNet (Shwartz et al., 2020; Vig et al., 2020; Yeo and Chen, 2020), though Bordia and Bowman (2019); Qian et al. (2019) also look at LSTM-based models.”
Persistent Anti-Muslim Bias in Large Language Models (2021, background): “Prior work has shown that language models can be debiased by preprocessing training datasets [7] or modifying the model training algorithm [10].”
Towards Understanding and Mitigating Social Biases in Language Models (2021, background): “Existing approaches towards mitigating biases in generation currently require retraining the models through adversarial trigger prompts (Sheng et al., 2020), data augmentation or collection (Dinan et al., 2020), and different objective functions (Qian et al., 2019; Huang et al., 2020).”
Queens Are Powerful Too: Mitigating Gender Bias in Dialogue Generation (2019, background): “2019), uncovering many gender disparities and harmful biases in algorithms and text (Cao and Daumé III 2020; Chang et al. 2019; Chang and McKeown 2019; Costa-jussà 2019; Du et al. 2019; Emami et al. 2019; Garimella et al. 2019; Gaut et al. 2020; Habash et al. 2019; Hashempour 2019; Hoyle et al. 2019”
Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias (2020, background): “…data (e.g., Lu et al., 2018; Hall Maudslay et al., 2019; Zhao et al., 2018a; Kaushik et al., 2019), the training process (e.g., Huang et al., 2019; Qian et al., 2019), or the model itself (e.g., Madras et al., 2019; Romanov et al., 2019; Gehrmann et al., 2019) to ensure counterfactual fairness.”
Multi-Dimensional Gender Bias Classification (2020, neutral): “, 2019), unigram language modeling (Qian et al., 2019), ap-”
RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models (2021, methodology): “…is surprisingly limited (Lee et al., 2019; Liu et al., 2020a,b; Dinan et al., 2020a,b), albeit some more research has recently emerged in the wider context of biases in general-purpose language generation models (Qian et al., 2019; Sheng et al., 2019; Nadeem et al., 2020; Yeo and Chen, 2020).”
Investigating Gender Bias in Language Models Using Causal Mediation Analysis (2020, background): “…data (e.g., Lu et al., 2018; Hall Maud-slay et al., 2019; Zhao et al., 2018a; Kaushik et al., 2019), the training process (e.g., Huang et al., 2019; Qian et al., 2019), or the model itself (e.g., Madras et al., 2019; Romanov et al., 2019; Gehrmann et al., 2019) to ensure counterfactual fairness.”
Towards Cross-Lingual Generalization of Translation Gender Bias (2021, background): “Accordingly, in purely text-based settings, many studies centered on evaluating the biasedness of gender-context correlation throughout the corpus [16] and mitigating it by, e.g., swapping the gender [21].”
Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting (2021, background): “Among various gender-biased behaviours of pre-trained models, bias on professions is the most prominent and wellstudied (de Vassimon Manela et al., 2021; Vig et al., 2020; Qian et al., 2019; Zhao et al., 2019).”
A Survey of Controllable Text Generation Using Transformer-based Pre-trained Language Models (2022, unknown): “Storystructure[32,33,40],storyending[85,132], topic[16,73,141,153],persona[15,76] DatatoText Input: [78],genderbias[28,104],subjectivebias[101], socialbias[4,121],sentimentbias[51],toxicity[62,74]) FormatControl Input:DesiredFormat,prompttext Output:Textinpre-definedformation Format[66,70,120,122]…”
CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Models (2023, methodology): “We briefly describe three loss-based debiasing methods: Language Model Debiasing (LMD): The additional loss is defined as: `bias = 1 |Pt| ∑ (ti,1,ti,2)⊂Pi ∣∣∣∣log ŷti,1ŷti,2 ∣∣∣∣ , where Pt is the target pairs set consisting of (ti,1, ti,2) pairs, and ti,1 ∈ T1, ti,2 ∈ T2; Pi ∈ Pt is one of target”
Directional Bias Amplification (2021, methodology): “Directionality of amplification has been observed (Stock & Cisse, 2018; Qian et al., 2019), but we take a more systematic approach.”
A Survey on Gender Bias in Natural Language Processing (2021, methodology): “Then, gender bias is defined as the disparity in the output when model is feeded with different genders [Qian et al. 2019].”