Whose Opinions Do Language Models Reflect?
By Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, Tatsunori B. Hashimoto. 2023
917 citations recorded by the source corpus.
Language models (LMs) are increasingly being used in open-ended contexts, where the opinions reflected by LMs in response to subjective queries can have a profound impact, both on user satisfaction, as well as shaping the views of society at large. In this work, we put forth a quantitative framework to investigate the opinions reflected by LMs -- by leveraging high-quality public opinion polls and their associated human responses. Using this framework, we create OpinionsQA, a new dataset for evaluating the alignment of LM opinions with those of 60 US demographic groups over topics ranging from abortion to automation. Across topics, we find substantial misalignment between the views reflected by current LMs and those of US demographic groups: on par with the Democrat-Republican divide on climate change. Notably, this misalignment persists even after explicitly steering the LMs towards particular demographic groups. Our analysis not only confirms prior observations about the left-leaning tendencies of some human feedback-tuned LMs, but also surfaces groups whose opinions are poorly reflected by current LMs (e.g., 65+ and widowed individuals). Our code and data are available at https://github.com/tatsu-lab/opinions_qa.
Subfields: Artificial Intelligence, General Social Sciences, Large Language Models, Evaluation & Benchmarks, AI Ethics, Fairness & Society
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Percy Liang
- Luke S. Zettlemoyer
- Jianfeng Gao
- Yejin Choi
- Sergey Levine
- Kyunghyun Cho
- Yoshua Bengio
In General Social Sciences
In Large Language Models
- Omar Shaikh
- Matei A. Zaharia
- Peter Hase
- James Y. Zou
- Anton Bakhtin
- Mario Fritz
- Diyi Yang
- Jared Kaplan
In Evaluation & Benchmarks
- Matei A. Zaharia
- Sahar Abdelnabi
- Caleb Ziems
- Peter Hase
- David Krueger
- James Y. Zou
- Thomas Liao
- Mario Fritz
In AI Ethics, Fairness & Society
Citations in the corpus
- Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback (2023, methodology): “Santurkar et al. [2023] found LLMs finetuned with RLHF expressed a narrow distribution of political views.”
- Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (2023, background): “Studying biases and the alignment between views expressed in LLM’s output and different demographic populations is already a pressing question [49, 70], with concerns that modelsmight bemore biased after human-feedback fine-tuning [70].”
- How is ChatGPT's behavior changing over time? (2023, background): “The stability of these modules’ prompts over time, i.e., how the performance of the same prompts changes over time, is therefore of interest to many users.”
- Towards Measuring the Representation of Subjective Global Opinions in Language Models (2023, positive): “Inspired by Santurkar et al. (2023) (see §6 for related work) we first compile a set of questions and responses from two established cross-national surveys designed to capture values and beliefs from thousands of participants across many countries: the Pew Global Attitudes Survey (PEW) 23 and the…”
- Can Large Language Models Transform Computational Social Science? (2023, background): “Additionally, current models exhibit higher homogeneity of opinions than humans (Argyle et al., 2022; Santurkar et al., 2023).”
- When LLMs Play the Telephone Game: Cultural Attractors as Conceptual Tools to Evaluate LLMs in Multi-turn Settings (2024, unknown): “LLMs trained through reinforcement learning with human feedback (RLHF) were found to overly express left-wing opinions on American politics — a tendency that, once formed, is difficult to avoid even after steering the model toward different demographic groups (Santurkar et al., 2023).”