Social Simulacra: Creating Populated Prototypes for Social Computing Systems

By Joon Sung Park, Lindsay Popowski, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, Michael S. Bernstein. ACM Symposium on User Interface Software and Technology, 2022

484 citations recorded by the source corpus.

Social computing prototypes probe the social behaviors that may arise in an envisioned system design. This prototyping practice is currently limited to recruiting small groups of people. Unfortunately, many challenges do not arise until a system is populated at a larger scale. Can a designer understand how a social system might behave when populated, and make adjustments to the design before the system falls prey to such challenges? We introduce social simulacra, a prototyping technique that generates a breadth of realistic social interactions that may emerge when a social computing system is populated. Social simulacra take as input the designer’s description of a community’s design—goal, rules, and member personas—and produce as output an instance of that design with simulated behavior, including posts, replies, and anti-social behaviors. We demonstrate that social simulacra shift the behaviors that they generate appropriately in response to design changes, and that they enable exploration of “what if?” scenarios where community members or moderators intervene. To power social simulacra, we contribute techniques for prompting a large language model to generate thousands of distinct community members and their social interactions with each other; these techniques are enabled by the observation that large language models’ training data already includes a wide variety of positive and negative behavior on social media platforms. In evaluations, we show that participants are often unable to distinguish social simulacra from actual community behavior and that social computing designers successfully refine their social computing designs when using social simulacra.

Subfields: Communication, Computer Networks and Communications, Sociology and Political Science

View Paper: https://www.semanticscholar.org/paper/49b499598a8864eee55ab264fc16a5bf8d2f87ef

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Communication

In Computer Networks and Communications

In Sociology and Political Science

Citations in the corpus

A survey on large language model based autonomous agents (2023, background): “For example, Social Simulacra [79] simulates an online social community and explores the potential of utilizing agent-based simulations to aid decision-makers to improve community regulations.”
Empowering Biomedical Discovery with AI Agents (2024, unknown): “Multi-agent interaction can also boost the proﬁciency of less skilled agents by allowing them to learn from more experienced counterparts [135].”
Whose Opinions Do Language Models Reflect? (2023, background): “Another set of studies have examined whether LMs can be used to simulate personas (Park et al., 2022; Argyle et al., 2022; Jiang et al., 2022; Simmons, 2022), akin to our notion of steerability.”
AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback (2023, background): “Our goal of emulating human annotators also connects to work on simulating humans with LMs based on personas [47, 46, 1, 3], as well as works that simulate human behavior in the context of cognitive science, social science, and economics [67, 25].”
Can Large Language Models Transform Computational Social Science? (2023, background): “Recently, this capacity of LLMs has been leveraged to simulate social computing systems (Park et al., 2022), community and their members’ interactions (Park et al., 2023), public opinion (Argyle et al., 2022; Chu et al., 2023), and subjective experience description (Argyle et al., 2022).”
Generative Agents: Interactive Simulacra of Human Behavior (2023, methodology): “The no observation, no refection, no planning condition efectively represents the previous state of the art for agents created through large language models [12, 46, 80].”
Character-LLM: A Trainable Agent for Role-Playing (2023, background): “Many attempts utilize prompted LLM (Wu et al., 2022a,b) that generate short natural language descriptions of per-sonas and the according behaviors, then they use the generated information to simulate social behaviors with language models (Park et al., 2022; Kim et al., 2022).”
Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies (2022, background): “…as proxies for a diverse set of humans, such as cheaply automating a variety of small writing tasks (Korinek, 2023) and using prompts to generate synthetic human-like interactions with desired properties (Park et al., 2022; Caron & Srivastava, 2022; Jiang et al., 2022; Karra et al., 2022, e.g.,).”
The rise and potential of large language model based agents: a survey (2025, unknown): “Techniques like prompt engineering [16,430] and personality-enriched datasets [431,432] are used to optimize LLMs and train them to exhibit speciﬁc personality traits.”
Evaluating Large Language Models in Generating Synthetic HCI Research Data: a Case Study (2023, methodology): “[59] have used GPT-3 to generate synthetic users and conversations for the purposes of prototyping social computing platforms, and Argyle et al.”
Harms from Increasingly Agentic Algorithmic Systems (2023, background): “Assuming either that we have the system in question or that we can simulate it faithfully [11, 63, 143], we can formulate and test hypotheses in simple experiments or simulation.”
Towards Measuring the Representation of Subjective Global Opinions in Language Models (2023, unknown): “…work has focused on mitigating known issues or aligning with clearly defined values, understanding how models function in settings involving ambiguity, nuance or diverse human experiences has been less explored (Karra et al., 2022; Park et al., 2022; Johnson et al., 2022; Arora et al., 2023).”
Large Language Model based Multi-Agents: A Survey of Progress and Challenges (2024, background): “…et al. , 2023; Qian et al. , 2023 ] , multi-robot systems [ Mandi et al. , 2023; Zhang et al. , 2023c ] , society simulation [ Park et al. , 2023; Park et al. , 2022 ] , policy simulation [ Xiao et al. , 2023; Hua et al. , 2023 ] , and game simulation [ Xu et al. , 2023c; Wang et al. , 2023c ] .”
Harnessing the Wisdom of LLM Crowds through Complementarity-Driven Iterative Collaboration (2026, unknown): “Such systems are typically tailored for specific tasks (like software development (Xiong et al. 2023), scientific experiments (Zheng et al. 2023), scientific debates (Tang et al. 2024, Du et al. 2023), social simulations (Park et al. 2022), etc.) and remain static once established.”
When LLMs Play the Telephone Game: Cultural Attractors as Conceptual Tools to Evaluate LLMs in Multi-turn Settings (2024, unknown): “…systems of LLMs may therefore be particularly relevant to situations in which LLMs are used to simulate artificial societies (de Zarzà et al., 2023; Park et al., 2023; 2022; Xiao et al., 2023; Hua et al., 2023; Vezhnevets et al., 2023; Chuang et al., 2023), where they are often granted a…”