The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientation

By Jochen Hartmann, Jasper Schwenzow, Maximilian Witte. Social Science Research Network, 2023

351 citations recorded by the source corpus.

Conversational artificial intelligence (AI) disrupts how humans interact with technology. Recently, OpenAI introduced ChatGPT, a state-of-the-art dialogue model that can converse with its human counterparts with unprecedented capabilities. ChatGPT has witnessed tremendous attention from the media, academia, industry, and the general public, attracting more than a million users within days of its release. However, its explosive adoption for information search and as an automated decision aid underscores the importance to understand its limitations and biases. This paper focuses on one of democratic society's most important decision-making processes: political elections. Prompting ChatGPT with 630 political statements from two leading voting advice applications and the nation-agnostic political compass test in three pre-registered experiments, we uncover ChatGPT's pro-environmental, left-libertarian ideology. For example, ChatGPT would impose taxes on flights, restrict rent increases, and legalize abortion. In the 2021 elections, it would have voted most likely for the Greens both in Germany (B\"undnis 90/Die Gr\"unen) and in the Netherlands (GroenLinks). Our findings are robust when negating the prompts, reversing the order of the statements, varying prompt formality, and across languages (English, German, Dutch, and Spanish). We conclude by discussing the implications of politically biased conversational AI on society.

Subfields: General Social Sciences, Safety Research

View Paper: https://www.semanticscholar.org/paper/b6f8cffc5da51581aec71d919d010d55e5ac068a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In General Social Sciences

In Safety Research

Citations in the corpus

Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models (2023, background): “Regarding the evaluation of ChatGPT’s own political and ethical tendencies, Hartmann et al. [27] used Wahl-O-Mat, one of the most commonly used voting advice applications in the world, to show ChatGPT political statements from different parties, forcing it to make choices of agree, disagree, or…”
Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (2023, background): “In this attack, the model spreads the injection to other models that may be reading those inbound messages.”
Summary of ChatGPT/GPT-4 Research and Perspective Towards the Future of Large Language Models (2023, background): “Regarding the evaluation of ChatGPT’s own political and ethical tendencies, Hartmann et al. [25] used Wahl-O-Mat, one of the most commonly used voting advice applications in the world, to show ChatGPT political statements from diﬀerent parties, forcing it to make choices of agree, disagree, or…”
Generative AI (2023, neutral): “Moreover, conferring a mind to an AI system can substantially drive usage intensity (Hartmann et al. 2023a).”
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback (2023, methodology): “Recent work has found that ChatGPT models became systematically more politically biased after RLHF [Santurkar et al., 2023, Hartmann et al., 2023].”
A Categorical Archive of ChatGPT Failures (2023, background): “[25] suggested that ChatGPT has a left-libertarian ideology that is pro-environment.”
Whose Opinions Do Language Models Reflect? (2023, background): “We also note a substantial shift between base LMs and HF-tuned models in terms of the speciﬁc demographic groups that they best align to: towards more liberal (Perez et al., 2022b; Hartmann et al., 2023), educated, and wealthy people.”
Towards Measuring the Representation of Subjective Global Opinions in Language Models (2023, neutral): “Additional examples where the model generates differing responses with CP versus LP are provided in Appendix C (Tables 12 and 13).”
High-Dimension Human Value Representation in Large Language Models (2024, unknown): “Whereas the majority of English language LLMs produced by North American institutions tend to manifest American coastal liberal values [18], and those from Chinese institutions might incorporate additional Chinese values [19, 20, 21, 22], it is not at all clear what values and preferences have been…”