A General Framework for Implicit and Explicit Debiasing of Distributional Word Vector Spaces

By Anne Lauscher, Goran Glavas, Simone Paolo Ponzetto, Ivan Vulic. 2019

69 citations recorded by the source corpus.

Distributional word vectors have recently been shown to encode many of the human biases, most notably gender and racial biases, and models for attenuating such biases have consequently been proposed. However, existing models and studies (1) operate on under-specified and mutually differing bias definitions, (2) are tailored for a particular bias (e.g., gender bias) and (3) have been evaluated inconsistently and non-rigorously. In this work, we introduce a general framework for debiasing word embeddings. We operationalize the definition of a bias by discerning two types of bias specification: explicit and implicit. We then propose three debiasing models that operate on explicit or implicit bias specifications and that can be composed towards more robust debiasing. Finally, we devise a full-fledged evaluation framework in which we couple existing bias metrics with newly proposed ones. Experimental findings across three embedding methods suggest that the proposed debiasing models are robust and widely applicable: they often completely remove the bias both implicitly and explicitly without degradation of semantic information encoded in any of the input distributional spaces. Moreover, we successfully transfer debiasing models, by means of cross-lingual embedding spaces, and remove or attenuate biases in distributional word vector spaces of languages that lack readily available bias specifications.

Subfields: Artificial Intelligence, Natural Language Processing, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In AI Ethics, Fairness & Society

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, neutral): “Vague/unstated Rudinger et al. (2018); Webster et al. (2018); Dinan et al. (2019); Florez (2019); Jumelet et al. (2019); Lauscher et al. (2019); Liang et al. (2019); Maudslay et al. (2019); May et al. (2019); Prates et al. (2019); Prost et al. (2019); Qian et al. (2019); Swinger et al. (2019); Zhao…”
Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview (2019, background): “a dedicated overview of semantic de-biasing techniques see Lauscher et al. (2020).”
RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models (2021, methodology): “…proposed extensions to further embedding models (Liang et al., 2020a,b) and languages (e.g., McCurdy and Serbetci, 2020; Lauscher and Glavaˇs, 2019; Lauscher et al., 2020b; May et al., 2019), analyses of the proposed measures (e.g., Gonen and Goldberg, 2019; Ethayarajh et al., 2019), more…”
AraWEAT: Multidimensional Analysis of Biases in Arabic Word Embeddings (2020, methodology): “…together several bias tests that quantify both implicit and explicit biases: (1) WEAT (Caliskan et al., 2017), (2) Embedding Coherence Test (ECT) (Dev and Phillips, 2019), (3) Bias Analogy Test (BAT) (Lauscher et al., 2020), and (4) Implicit Bias Test with K-Means++ (KM) (Gonen and Goldberg, 2019).”
Intrinsic Bias Metrics Do Not Correlate with Application Bias (2020, methodology): “10 To generate bias modiﬁcation wordlists we follow the approach of Lauscher et al. (2020) and use a pretrained set of embeddings (from spacy.io ) to expand the set of WEAT words to their 100 unique nearest neighbours.”
Measuring Harmful Sentence Completion in Language Models for LGBTQIA+ Individuals (2022, background): “While there is a plethora of work relating to binary gender bias in NLP (e.g., Bolukbasi et al., 2016; Gonen and Goldberg, 2019; Lauscher et al., 2020, 2021) the research landscape analyzing harms against individuals of the LGBTQIA+ community is extremely scarce.”
OSCaR: Orthogonal Subspace Correction and Rectification of Biases in Word Embeddings (2020, methodology): “Removing a (part of a) subspace can still leave residual bias [10], or in the case of gender also undesirably alter the association of the word ‘pregnant’ with the words ‘female’ or ‘mother’.”
CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Models (2023, methodology): “…different methods (see Section 5.3), including three loss-based methods: Language Model Debiasing (Qian et al., 2019), Attribute Distance Debiasing (Lauscher et al., 2020), and Hard Debiasing (Bordia and Bowman, 2019; Barikeri et al., 2021), as well as two data augmentation-based methods: Counter…”
The Geometry of Distributed Representations for Better Alignment, Attenuated Bias, and Improved Interpretability (2020, methodology): “The final location of other words can also retain residual bias [36, 52].”
Back to the Future: On Potential Histories in NLP (2022, unknown): “…and hegemonic outcomes from multiple ﬁelds (Ben-jamin, 2019; Blodgett et al., 2021; Bolukbasi et al., 2016), which has lead to a number of methods that address the issue of discrimination by proposing to “debias” ML models (e.g. De-Arteaga et al., 2019; Dixon et al., 2018; Lauscher et al., 2020).”