Stepmothers are mean and academics are pretentious: What do pretrained language models learn about you?
By Rochelle Choenni, Ekaterina Shutova, Robert van Rooij. 2021
32 citations recorded by the source corpus.
In this paper, we investigate what types of stereotypical information are captured by pretrained language models. We present the first dataset comprising stereotypical attributes of a range of social groups and propose a method to elicit stereotypes encoded by pretrained language models in an unsupervised fashion. Moreover, we link the emergent stereotypes to their manifestation as basic emotions as a means to study their emotional effects in a more generalized manner. To demonstrate how our methods can be used to analyze emotion and stereotype shifts due to linguistic experience, we use fine-tuning on news sources as a case study. Our experiments expose how attitudes towards different social groups vary across models and how quickly emotions and stereotypes can shift at the fine-tuning stage.
Subfields: Artificial Intelligence, General Social Sciences, Natural Language Processing, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Hailey Schoelkopf
- Lintang Sutawika
- USVSN Sai Prashanth
- Shivanshu Purohit
- Quentin Anthony
- Stella Biderman
- Oskar van der Wal
- Eric Hallahan
In General Social Sciences
In Natural Language Processing
In Safety, Robustness & Interpretability
Citations in the corpus
- Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling (2023, background): “However, while some work has explored finetuning’s effects on bias in language models (Gira et al., 2022; Kirtane et al., 2022; Choenni et al., 2021), or the relationship between the corpus statistics and the measured bias (Bordia & Bowman, 2019; Van der Wal et al.”
- The Birth of Bias: A case study on the evolution of gender bias in an English language model (2022, unknown): “Indeed, work on studying the effect of fine-tuning has shown that the manifestation of bias can still change significantly in pre-trained models (Choenni et al., 2021; Webster et al., 2020).”