Biased Embeddings from Wild Data: Measuring, Understanding and Removing

By Adam Sutton, Thomas Lansdall-Welfare, Nello Cristianini. 2018

25 citations recorded by the source corpus.

Many modern Artificial Intelligence (AI) systems make use of data embeddings, particularly in the domain of Natural Language Processing (NLP). These embeddings are learnt from data that has been gathered “from the wild” and have been found to contain unwanted biases. In this paper we make three contributions towards measuring, understanding and removing this problem. We present a rigorous way to measure some of these biases, based on the use of word lists created for social psychology applications; we observe how gender bias in occupations reflects actual gender bias in the same occupations in the real world; and finally we demonstrate how a simple projection can significantly reduce the effects of embedding bias. All this is part of an ongoing effort to understand how trust can be built into AI systems.

Subfields: Artificial Intelligence, Safety Research, Sociology and Political Science, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Hal Daumé Iii
2. Houda Bouamor
3. Nizar Habash
4. Su Lin Blodgett
5. Hanna M. Wallach
6. Ryan Cotterell
7. Henrik Björklund
8. Hannah Devinney

In Safety Research

1. Hanna M. Wallach
2. Su Lin Blodgett
3. Hal Daumé Iii
4. Ryan Cotterell
5. Solon Barocas
6. Adrian Weller
7. Alexandra Chouldechova
8. Berk Ustun

In Sociology and Political Science

1. Su Lin Blodgett
2. Nizar Habash
3. Hal Daumé Iii
4. Ryan Cotterell
5. Hanna M. Wallach
6. Solon Barocas

In Representation & Self-Supervised Learning

1. Nizar Habash
2. Hal Daumé Iii
3. Ryan Cotterell
4. Hanna M. Wallach
5. Houda Bouamor
6. Kate Saenko
7. Qiang Yang
8. Mohammad Rostami

Citations in the corpus

• Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “…(2017); McCurdy and Serbetçi (2017); Rudinger et al. (2017); Zhao et al. (2017); Curry and Rieser (2018); Díaz et al. (2018); Santana et al. (2018); Sutton et al. (2018); Zhao et al. (2018a,b); Agarwal et al. (2019); Basta et al. (2019); Bhaskaran and Bhallamudi (2019); Bordia and Bowman (2019);…”
• Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology (2019, background): “Gender stereotypes of this sort have been observed in word embeddings (Bolukbasi et al., 2016; Sutton et al., 2018), contextual word embeddings (Zhao et al.”
• Automatic Gender Identification and Reinflection in Arabic (2019, background): “word embeddings in a number of studies (Bolukbasi et al., 2016; Caliskan et al., 2017; Sutton et al., 2018; Basta et al., 2019; Garg et al., 2018; Zhao et al., 2018, 2019).”
• Theories of “Gender” in NLP Bias Research (2022, background): “08976 [141] Adam Sutton, Thomas Lansdall-Welfare, and Nello Cristianini.”