A Causal Inference Method for Reducing Gender Bias in Word Embedding Relations

By Zekun Yang, Juan Feng. 2019

41 citations recorded by the source corpus.

Word embedding has become essential for natural language processing as it boosts empirical performances of various tasks. However, recent research discovers that gender bias is incorporated in neural word embeddings, and downstream tasks that rely on these biased word vectors also produce gender-biased results. While some word-embedding gender-debiasing methods have been developed, these methods mainly focus on reducing gender bias associated with gender direction and fail to reduce the gender bias presented in word embedding relations. In this paper, we design a causal and simple approach for mitigating gender bias in word vector relation by utilizing the statistical dependency between gender-definition word embeddings and gender-biased word embeddings. Our method attains state-of-the-art results on gender-debiasing tasks, lexical- and sentence-level evaluation tasks, and downstream coreference resolution tasks.

Subfields: Artificial Intelligence, Causal Inference & Discovery, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yonatan Belinkov
2. Sebastian Gehrmann
3. Jesse Vig
4. Stuart M. Shieber
5. Sharon Qian
6. Daniel Nevo
7. Yaron Singer
8. Yoav Goldberg

In Causal Inference & Discovery

1. Stuart M. Shieber
2. Yaron Singer
3. Jesse Vig
4. Yonatan Belinkov
5. Sharon Qian
6. Sebastian Gehrmann
7. Daniel Nevo

In AI Ethics, Fairness & Society

1. Yonatan Belinkov
2. Sebastian Gehrmann

Citations in the corpus

• Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, neutral): “…al. (2020); Nadeem et al. (2020); Nissim et al. (2020); Papakyriakopoulos et al. (2020); Popović et al. (2020); Ravfogel et al. (2020); Ross et al. (2020); Rozado (2020); Saunders and Byrne (2020); Shin et al. (2020); Vig et al. (2020); Wang et al. (2020); Yang and Feng (2020); Zhao et al. (2020).”
• Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias (2020, background): “ual probabilities across grammatical genders in referential expressions. Others have focused on de-biasing word embeddings and contextual word representations (Bolukbasi et al.,2016;Zhao et al.,2018b;Yang and Feng, 2020), though recent work has questioned the efficacy of these debiasing techniques in”
• Theories of “Gender” in NLP Bias Research (2022, unknown): “org/anthology/W19-3815/ [161] Zekun Yang and Juan Feng.”
• Investigating Gender Bias in Language Models Using Causal Mediation Analysis (2020, background): “Others have focused on de-biasing word embeddings and contextual word representations (Bolukbasi et al., 2016; Zhao et al., 2018b; Yang and Feng, 2020), though recent work has questioned the efficacy of these debiasing techniques (Elazar and Goldberg, 2018; Gonen and Goldberg, 2019).”
• Leveraging Bias in Pre-Trained Word Embeddings for Unsupervised Microaggression Detection (2021, unknown): “Many studies have attempted to reduce this bias (Yang and Feng, 2020; Zhao et al., 2018; Manzini et al., 2019).”