Role play with large language models

By Murray Shanahan, Kyle McDonell, Laria Reynolds. Nature, 2023

609 citations recorded by the source corpus.

By casting large-language-model-based dialogue-agent behaviour in terms of role play, it is possible to describe dialogue-agent behaviour such as (apparent) deception and (apparent) self-awareness without misleadingly ascribing human characteristics to the models. As dialogue agents become increasingly human-like in their performance, we must develop effective ways to describe their behaviour in high-level terms without falling into the trap of anthropomorphism. Here we foreground the concept of role play. Casting dialogue-agent behaviour in terms of role play allows us to draw on familiar folk psychological terms, without ascribing human characteristics to language models that they in fact lack. Two important cases of dialogue-agent behaviour are addressed this way, namely, (apparent) deception and (apparent) self-awareness.

Subfields: Artificial Intelligence, Social Psychology

View Paper: https://www.semanticscholar.org/paper/aa23bcf357fc4f890e0a97c27e254d14fbacd460

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Social Psychology

Citations in the corpus

SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents (2023, background): “This role-playing setting discourages AI systems with consistent human personalities, which could lead to anthropomorphism (Shanahan et al., 2023).”
Prompt Injection attack against LLM-integrated Applications (2023, background): “Jailbreak [33,55,58,60] involves eliciting model-generated content that divulges training data specifics, which can lead to serious privacy breaches, particularly when training data include sensitive or private information.”
Large language models empowered agent-based modeling and simulation: a survey and perspectives (2023, neutral): “That is, LLM agents should be able to comprehend, perceive, and respond to diverse needs, emotions, and attitudes within different contexts, from the “ ﬁ rst-view sight ” (Shanahan et al., 2023).”
Scaling Synthetic Data Creation with 1,000,000,000 Personas (2024, background): “…specific data synthesis prompts, personas can be combined with almost any data synthesis prompt, benefiting from an LLM’s strong roleplay ability (Shanahan et al., 2023; Li et al., 2023a; Choi & Li, 2024; Wang et al., 2024), making them generally applicable to a variety of data synthesis…”
Refusal in Language Models Is Mediated by a Single Direction (2024, neutral): “One hypothesis for why this works is that such prompts modify the LLM assistant’s "persona" (Andreas, 2022; Park et al., 2023a; Shanahan et al., 2023).”
Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review (2023, background): “Contemporary prompt engineering encompasses a spectrum of techniques, ranging from foundational approaches such as role-prompting [15] to more sophisticated methods such as “chain of thought” prompting [16].”
When LLMs Play the Telephone Game: Cultural Attractors as Conceptual Tools to Evaluate LLMs in Multi-turn Settings (2024, unknown): “Moreover, our framework enables us to avoid assuming that LLMs possess “priors” in the same way that humans do, which might be problematic (see Kova ˇ c et al. (2023); Shanahan et al. (2023)).”