Fair Is Better than Sensational: Man Is to Doctor as Woman Is to Doctor

By Malvina Nissim, Rik van Noord, Rob van der Goot. 2019

111 citations recorded by the source corpus.

Analogies such as man is to king as woman is to X are often used to illustrate the amazing power of word embeddings. Concurrently, they have also been used to expose how strongly human biases are encoded in vector spaces trained on natural language, with examples like man is to computer programmer as woman is to homemaker. Recent work has shown that analogies are in fact not an accurate diagnostic for bias, but this does not mean that they are not used anymore, or that their legacy is fading. Instead of focusing on the intrinsic problems of the analogy task as a bias detection tool, we discuss a series of issues involving implementation as well as subjective choices that might have yielded a distorted picture of bias in word embeddings. We stand by the truth that human biases are present in word embeddings, and, of course, the need to address them. But analogies are not an accurate tool to do so, and the way they have been most often used has exacerbated some possibly non-existing biases and perhaps hidden others. Because they are still widely popular, and some of them have become classics within and outside the NLP community, we deem it important to provide a series of clarifications that should put well-known, and potentially new analogies, into the right perspective.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

Language Models are Few-Shot Learners (2020, background): “Thus, mitigation work should not be approached purely with a metric driven objective to ‘remove’ bias as this has been shown to have blind spots [GG19, NvNvdG19] but in a holistic manner.”
Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, neutral): “…(2019); Lauscher and Glavaš (2019); Loukina et al. (2019); Mayfield et al. (2019); Mirzaev et al. (2019); Prabhumoye et al. (2019); Ruane et al. (2019); Sedoc and Ungar (2019); Sun et al. (2019); Nissim et al. (2020); Rozado (2020); Shah et al. (2020); Strengers et al. (2020); Wright et al. (2020).”
Release Strategies and the Social Impacts of Language Models (2019, background): “Training data helps define feature embeddings in the model and dataset selection conditions the model’s displayed biases [43].”
On the Binding Problem in Artificial Neural Networks (2020, background): “…Hinton et al., 2012; Krizhevsky et al., 2012) and are known to partially capture the semantic structure of a task (Figure 5), such as interpretable image features (Zeiler and Fergus, 2014; Olah et al., 2020), or the semantic structure of text (Mikolov et al., 2013; but compare Nissim et al., 2019).”
Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview (2019, methodology): “, 2013), a method whose informativeness has since been questioned, though (Nissim et al., 2019).”
Inducing Relational Knowledge from BERT (2019, background): “For instance, Rubinstein et al. (2015) analyzed how well pretrained Skip-gram and GloVe vectors are able to predict properties of nouns, focusing on both taxonomic properties (e.g. being a bird) and attributive properties (e.g. being dangerous).”
BERTIN: Efficient Pre-Training of a Spanish Language Model using Perplexity Sampling (2022, methodology): “We performed a basic ad-hoc bias analysis looking into possible shortcomings of our models (Nissim, van Noord, and van der Goot, 2020; Blodgett et al., 2020; Aka et al., 2021; Bender et al., 2021).”
Measuring Social Bias in Knowledge Graph Embeddings (2019, background): “Despite (Nissim et al., 2019) exposing some technical shortcomings in this approach, it remains the case that distance metrics appear to be appropriate in at least exposing bias in word embeddings, which has then been shown to clearly propagate to downstream tasks, (Zhao et al., 2018a; Stanovsky et…”
Theories of “Gender” in NLP Bias Research (2022, background): “[102] Malvina Nissim, Rik van Noord, and Rob van der Goot.”
Towards Cross-Lingual Generalization of Translation Gender Bias (2021, background): “Up-to-date studies claimed that such kind of experiment could not represent the bias in its entirety [18], but the tendency is still preserved in that the learning-based NLP frameworks integrate corpus-level bias projected onto it.”
Five sources of bias in natural language processing (2021, background): “There has been some justified scepticism over whether these analogy tasks are the best way to evaluate embedding models (Nissim et al., 2020), but there is plenty of evidence that (1) embed-dings do capture societal attitudes (Bhatia, 2017; Garg et al., 2018; Kozlowski et al., 2019), and that (2)…”
Cross-Loss Influence Functions to Explain Deep Network Representations (2020, background): “Bolukbasi et al. (2016) were among the ﬁrst to discover the inherent biases in word embeddings, a ﬁnding that has been replicated in transformers (Basta et al., 2019; Kurita et al., 2019; Silva et al., 2021) and more (Costa-jussà et al., 2019; Swinger et al., 2019; Nissim et al., 2020).”
Undesirable biases in NLP: Averting a crisis of measurement (2022, background): “Many problems have been pointed out for how bias is defined and operationalized (see e.g., Blodgett et al., 2020, 2021; Dev et al., 2022; Ethayarajh et al., 2019; Gonen & Goldberg, 2019; Nissim et al., 2020; Talat et al., 2022).”
The Political Biases of ChatGPT (2023, background): “There is also preliminary evidence that some concerns about algorithmic bias might have been exaggerated, generating in the process unwarranted sensationalism (Nissim et al. 2019).”