Human-level play in the game of Diplomacy by combining language models with strategic reasoning
By Anton Bakhtin, Noam Brown, Emily Dinan, Gabriele Farina, Colin Flaherty, Daniel Fried, A. Goff, Jonathan Gray, Hengyuan Hu, Athul Paul Jacob and 16 more. 2022
592 citations recorded by the source corpus.
Despite much progress in training artificial intelligence (AI) systems to imitate human language, building agents that use language to communicate intentionally with humans in interactive environments remains a major challenge. We introduce Cicero, the first AI agent to achieve human-level performance in Diplomacy, a strategy game involving both cooperation and competition that emphasizes natural language negotiation and tactical coordination between seven players. Cicero integrates a language model with planning and reinforcement learning algorithms by inferring players’ beliefs and intentions from its conversations and generating dialogue in pursuit of its plans. Across 40 games of an anonymous online Diplomacy league, Cicero achieved more than double the average score of the human players and ranked in the top 10% of participants who played more than one game. Description AI masters Diplomacy The game Diplomacy has been a major challenge for artificial intelligence (AI). Unlike other competitive games that AI has recently mastered, such as chess, Go, and poker, Diplomacy cannot be solved purely through self-play; it requires the development of an agent to understand other players’ motivations and perspectives and to use natural language to negotiate complex shared plans. The Meta Fundamental AI Research Diplomacy Team (FAIR) et al. developed an agent that is able to play the full natural language form of the game and demonstrates performance well above the human average in an online Diplomacy league. The present work has far-reaching implications for the development of cooperative AI and language models for communication with people, even when interactions involve a mixture of aligned and competing interests. —YS Artificial intelligence demonstrates human-level performance in the strategic board game Diplomacy.
Subfields: Artificial Intelligence, LLM Agents & Tool Use, Reinforcement Learning, Game & Decision Theory
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Percy Liang
- Mohit Bansal
- Luke S. Zettlemoyer
- Quoc V. Le
- Jianfeng Gao
- Dan Hendrycks
- Yang Liu
- Graham Neubig
In LLM Agents & Tool Use
In Reinforcement Learning
In Game & Decision Theory
Citations in the corpus
- The alignment problem from a deep learning perspective (2022, background): “These networks have reached human-level performance in complex games like StarCraft 2 [Vinyals et al., 2019] and Diplomacy [Bakhtin et al., 2022], while also exhibiting growing generality [Bom-masani et al., 2021] through improvements in areas including sample efficiency [Brown et al., 2020, Dorner,…”
- Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf (2023, background): “, 2023) or require a significant amount of human-annotated data (FAIR et al., 2022; Kramár et al., 2022).”
- An Overview of Catastrophic AI Risks (2023, background): “In 2022, Meta AI revealed an agent called CICERO, which was trained to play a game called Diplomacy [118].”
- Discovering Latent Knowledge in Language Models Without Supervision (2022, background): “To illustrate this, imagine we train reinforcement learning agents to play a game such as Diplomacy (FAIR et al., 2022), in which players have incentives to lie to each other.”
- Can We Edit Factual Knowledge by In-Context Learning? (2023, background): “Due to their promising results, researchers have empowered LMs with new skills that meet real-world needs, such as using web browsers (Nakano et al., 2021), coding (Chen et al., 2021), playing strategic game (FAIR et al., 2022), and conversational talents (OpenAI, 2022, 2023).”
- Mind the Gap! Choice Independence in Using Multilingual LLMs for Persuasive Co-Writing Tasks in Different Languages (2025, unknown): “LLM-generated text has been shown to influence political attitudes [116], vaccine uptake [66], strategic negotiations [32], personal beliefs [100], or even romantic conversations [141].”