It’s All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitution

By Rowan Hall Maudslay, Hila Gonen, Ryan Cotterell, Simone Teufel. 2019

202 citations recorded by the source corpus.

This paper treats gender bias latent in word embeddings. Previous mitigation attempts rely on the operationalisation of gender bias as a projection over a linear subspace. An alternative approach is Counterfactual Data Augmentation (CDA), in which a corpus is duplicated and augmented to remove bias, e.g. by swapping all inherently-gendered words in the copy. We perform an empirical comparison of these approaches on the English Gigaword and Wikipedia, and find that whilst both successfully reduce direct bias and perform well in tasks which quantify embedding quality, CDA variants outperform projection-based methods at the task of drawing non-biased gender analogies by an average of 19% across both corpora. We propose two improvements to CDA: Counterfactual Data Substitution (CDS), a variant of CDA in which potentially biased text is randomly substituted to avoid duplication, and the Names Intervention, a novel name-pairing technique that vastly increases the number of words being treated. CDA/S with the Names Intervention is the only approach which is able to mitigate indirect gender bias: following debiasing, previously biased words are significantly less clustered according to gender (cluster purity is reduced by 49%), thus improving on the state-of-the-art for bias mitigation.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, neutral): “…et al. (2018); Webster et al. (2018); Dinan et al. (2019); Florez (2019); Jumelet et al. (2019); Lauscher et al. (2019); Liang et al. (2019); Maudslay et al. (2019); May et al. (2019); Prates et al. (2019); Prost et al. (2019); Qian et al. (2019); Swinger et al. (2019); Zhao et al. (2019);…”
Fair Is Better than Sensational: Man Is to Doctor as Woman Is to Doctor (2019, background): “One is that they get easily propagated both in science itself (Jha and Mamidi 2017; Gebru et al. 2018; Mohammad et al. 2018; Hall Maudslay et al. 2019), also outside NLP and artiﬁcial intelligence (McQuillan 2018) and in popularized articles (Zou and Schiebinger 2018), where readers are usually in n”
Re-contextualizing Fairness in NLP: The Case of India (2022, background): “Bias evaluation in NLP relies on proxies of subgroups in language, such as identity terms and personal names, to reveal the undesirable associations present in models and data (Caliskan et al., 2017; Maudslay et al., 2019).”
Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue models (2021, methodology): “…we compare and contrast the effectiveness of several mitigation strategies, including counterfactual data augmentation (Lu et al., 2018; Hall Maudslay et al., 2019) to scramble names, a novel application of unlikelihood training to bias mitigation (Welleck et al., 2019b), and controlled…”
Queens Are Powerful Too: Mitigating Gender Bias in Dialogue Generation (2019, background): “A straightforward solution for gender bias in embeddings is Counterfactual Data Augmentation (CDA) (Hall Maudslay et al., 2019; Zmigrod et al., 2019; Liu et al., 2019).”
Does Gender Matter? Towards Fairness in Dialogue Systems (2019, methodology): “Borrowing the idea from (Maudslay et al., 2019), we simply augment the training data by adding counterpart dialogue data based on the original data.”
Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias (2020, background): “A common strategy to mitigate biases is to change the training data (e.g., Lu et al., 2018; Hall Maudslay et al., 2019; Zhao et al., 2018a; Kaushik et al., 2019), the training process (e.g., Huang et al., 2019; Qian et al., 2019), or the model itself (e.g., Madras et al., 2019; Romanov et al., 2019;”
Learning the Difference that Makes a Difference with Counterfactually-Augmented Data (2019, methodology): “[19] describe a method for probabilistic automatic in-place substitution of gendered words in a corpus.”
Measuring and Reducing Gendered Correlations in Pre-trained Models (2020, positive): “…strategies have become popular recently, in particular re-balancing (Dixon et al., 2018) and counterfactual data augmentation (CDA), which augments training data using controlled perturbations to names or demographic attributes (Hall Maudslay et al., 2019; Zhao et al., 2019; Zmigrod et al., 2019).”
Towards Understanding Gender Bias in Relation Extraction (2019, background): “…by replacing masculine words in a sentence with their corresponding feminine words and vice versa for all sentences in a corpus, and then training on the union of the original and augmented corpora 3 (Zhao et al., 2018; Lu et al., 2018; Dixon et al., 2018; Maudslay et al., 2019; Zhao et al., 2019).”
On the Relationships Between the Grammatical Genders of Inanimate Nouns and Their Co-Occurring Adjectives and Verbs (2020, unknown): “…computational techniques for mitigating gender norms and stereotypes in a wide range of languagebased applications (Dev and Phillips, 2019; Dinan et al., 2019; Ethayarajh et al., 2019; Hall Maudslay et al., 2019; Stanovsky et al., 2019; Tan and Celis, 2019; Zhou et al., 2019; Zmigrod et al., 2019).”
The practical ethics of bias reduction in machine translation: why domain adaptation is better than data debiasing (2021, background): “Such approaches exist for English data (e.g., Maudslay et al. 2019; Zhao et al., 2018).”
Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond (2021, methodology): “Counterfactual data augmentation has also been applied to reduce bias in pre-trained models (e.g., Huang et al., 2019; Maudslay et al., 2019) but the extent to which biases in pre-trained models propagate to down-stream applications remains unclear (Goldfarb-Tarrant et al., 2021).”
Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics (2021, unknown): “…(Bolukbasi et al., 2016; Caliskan et al., 2017; Kurita et al., 2019; Sedoc and Ungar, 2019; Chaloner and Maldonado, 2019; Dev and Phillips, 2019; Gonen and Goldberg, 2019; Hall Maudslay et al., 2019; Liang et al., 2020; Shin et al., 2020; Liang et al., 2020; Papakyriakopoulos et al., 2020).”
AraWEAT: Multidimensional Analysis of Biases in Arabic Word Embeddings (2020, background): “While it has been shown that names are a good proxy for identifying and removing bias towards specific groups of people (Hall Maudslay et al., 2019), it is difficult to “translate” them.3 As an example of the resulting AraWEAT test, Table 1 list the Arabic translation of WEAT test T7.”
The Birth of Bias: A case study on the evolution of gender bias in an English language model (2022, unknown): “Many recent papers have proposed methods that work at one or two of these loci, for example, by focusing on the dataset (Dixon et al., 2018; Hall Maudslay et al., 2019; Lu et al.)”