Gender bias and stereotypes in Large Language Models
By Hadas Kotek, Rikker Dockum, David Q. Sun. International Conference on Climate Informatics, 2023
460 citations recorded by the source corpus.
Large Language Models (LLMs) have made substantial progress in the past several months, shattering state-of-the-art benchmarks in many domains. This paper investigates LLMs’ behavior with respect to gender stereotypes, a known issue for prior models. We use a simple paradigm to test the presence of gender bias, building on but differing from WinoBias, a commonly used gender bias dataset, which is likely to be included in the training data of current LLMs. We test four recently published LLMs and demonstrate that they express biased assumptions about men and women’s occupations. Our contributions in this paper are as follows: (a) LLMs are 3-6 times more likely to choose an occupation that stereotypically aligns with a person’s gender; (b) these choices align with people’s perceptions better than with the ground truth as reflected in official job statistics; (c) LLMs in fact amplify the bias beyond what is reflected in perceptions or the ground truth; (d) LLMs ignore crucial ambiguities in sentence structure 95% of the time in our study items, but when explicitly prompted, they recognize the ambiguity; (e) LLMs provide explanations for their choices that are factually inaccurate and likely obscure the true reason behind their predictions. That is, they provide rationalizations of their biased behavior. This highlights a key property of these models: LLMs are trained on imbalanced datasets; as such, even with the recent successes of reinforcement learning with human feedback, they tend to reflect those imbalances back at us. As with other types of societal biases, we suggest that LLMs must be carefully tested to ensure that they treat minoritized individuals and communities equitably.
Subfields: Artificial Intelligence, Health Informatics
View Paper: https://www.semanticscholar.org/paper/cf852c18387cfcdcb3cb3bb5ba6a549b35766c33
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
Citations in the corpus
- A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly (2023, background): “Concerns of bias were raised from various fields, encompassing gender and minority groups [64, 139, 80, 235], the identification of misinformation, political aspects.”
- Tree of Attacks: Jailbreaking Black-Box LLMs Automatically (2023, background): “An exciting open problem is to use fine-tuned open-source LLMs as evaluator to achieve a higher success rate than with GPT4 as the evaluator.”
- Rethinking machine unlearning for large language models (2024, unknown): “These include societal biases and stereotyping Bender et al. (2021); Motoki et al. (2023); Kotek et al. (2023), the generation of sensitive, private, harmful, or illegal content Nasr et al. (2023); Wen et al. (2023); Karamolegkou et al. (2023); Patil et al. (2024), ease of jailbreaking Wei et al.…”
- A Survey on the Memory Mechanism of Large Language Model-based Agents (2024, unknown): “Gödel Agent [206] utilizes dynamic code modification techniques to alter its own logic and behavior based on the feedback signals from the environment and stores the updated agent logic in the agent memory for recursive improvement.”
- When LLMs Play the Telephone Game: Cultural Attractors as Conceptual Tools to Evaluate LLMs in Multi-turn Settings (2024, unknown): “In single-turn settings, LLMs perpetrate (Nadeem et al., 2020) or even amplify human biases based on gender, nationality, race, and religion (Kotek et al., 2023).”