Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models

By Myra Cheng, Esin Durmus, Dan Jurafsky. Annual Meeting of the Association for Computational Linguistics, 2023

323 citations recorded by the source corpus.

To recognize and mitigate harms from large language models (LLMs), we need to understand the prevalence and nuances of stereotypes in LLM outputs. Toward this end, we present Marked Personas, a prompt-based method to measure stereotypes in LLMs for intersectional demographic groups without any lexicon or data labeling.Grounded in the sociolinguistic concept of markedness (which characterizes explicitly linguistically marked categories versus unmarked defaults), our proposed method is twofold: 1) prompting an LLM to generate personas, i.e., natural language descriptions, of the target demographic group alongside personas of unmarked, default groups; 2) identifying the words that significantly distinguish personas of the target group from corresponding unmarked ones.We find that the portrayals generated by GPT-3.5 and GPT-4 contain higher rates of racial stereotypes than human-written portrayals using the same prompts. The words distinguishing personas of marked (non-white, non-male) groups reflect patterns of othering and exoticizing these demographics. An intersectional lens further reveals tropes that dominate portrayals of marginalized groups, such as tropicalism and the hypersexualization of minoritized women. These representational harms have concerning implications for downstream applications like story generation.

Subfields: Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/8d9ca1e2c703e2752a4904c967a65d45d0bef5f6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Human-Computer Interaction

1. Maarten Sap
2. Louis-philippe Morency
3. Amir Zadeh
4. Percy Liang
5. Soujanya Poria

Citations in the corpus

• Coarse race data conceals disparities in clinical risk score performance (2023, background): “…et al., 2022; Laufer et al., 2022; Pierson, 2020; Liu and Garg, 2022; Derenoncourt and Montialoux, 2021; Chouldechova, 2017; Garg et al., 2018; Cheng et al., 2023; Bianchi et al., 2023; Abdu et al., 2023), suggesting the importance of examining granular race categories in these domains as…”
• Towards Measuring the Representation of Subjective Global Opinions in Language Models (2023, unknown): “Additional examples where the model generates differing responses with CP versus LP are provided in Appendix C (Tables 12 and 13).”
• SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents (2023, background): “As LLMs are becoming more and more popular in simulating human social interactions, there are also works that focus on investigating the potential issues and challenges of using LLMs in social simulations, such as stereotypes and reporting issues (Cheng et al., 2023b; Zhou et al., 2024).”