Superhuman AI for multiplayer poker
By Noam Brown, Tuomas Sandholm. 2019
884 citations recorded by the source corpus.
AI now masters six-player poker Computer programs have shown superiority over humans in two-player games such as chess, Go, and heads-up, no-limit Texas hold'em poker. However, poker games usually include six players—a much trickier challenge for artificial intelligence than the two-player variant. Brown and Sandholm developed a program, dubbed Pluribus, that learned how to play six-player no-limit Texas hold'em by playing against five copies of itself (see the Perspective by Blair and Saffidine). When pitted against five elite professional poker players, or with five copies of Pluribus playing against one professional, the computer performed significantly better than humans over the course of 10,000 hands of poker. Science, this issue p. 885; see also p. 864 An AI dubbed Pluribus performs significantly better than human professionals in six-player no-limit Texas hold’em poker. In recent years there have been great strides in artificial intelligence (AI), with games often serving as challenge problems, benchmarks, and milestones for progress. Poker has served for decades as such a challenge problem. Past successes in such benchmarks, including poker, have been limited to two-player games. However, poker in particular is traditionally played with more than two players. Multiplayer games present fundamental additional issues beyond those in two-player games, and multiplayer poker is a recognized AI milestone. In this paper we present Pluribus, an AI that we show is stronger than top human professionals in six-player no-limit Texas hold’em poker, the most popular form of poker played by humans.
Subfields: Artificial Intelligence, Clinical Psychology, Sociology and Political Science, Reinforcement Learning, Game & Decision Theory
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Yoshua Bengio
- Pieter Abbeel
- Ruslan Salakhutdinov
- Mohit Bansal
- Quoc V. Le
- Percy Liang
- Chelsea Finn
In Clinical Psychology
- Neil Burch
- Jakob N. Foerster
- Hengyuan Hu
- Zhuoran Yang
- José Hernandez-Orallo
- Adam Lerer
- Michal Kosinski
- Michael H. Bowling
In Sociology and Political Science
- Noam Brown
- Jakob N. Foerster
- Neil Burch
- Hengyuan Hu
- Michael H. Bowling
- Adam Lerer
- Viliam Lisý
- Michal Kosinski
In Reinforcement Learning
- Sergey Levine
- Ofir Nachum
- Jakob N. Foerster
- Pieter Abbeel
- Aviral Kumar
- Shimon Whiteson
- Chelsea Finn
- Adam Lerer
In Game & Decision Theory
- Noam Brown
- Neil Burch
- Adam Lerer
- Jakob N. Foerster
- Zhuoran Yang
- Hengyuan Hu
- Viliam Lisý
- Michael H. Bowling
Citations in the corpus
- OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learning (2020, background): “…Learning (RL) systems have achieved impressive performance in a variety of on-line settings such as games (Silver et al., 2016; Tesauro, 1995; Brown & Sandholm, 2019) and robotics (Levine et al., 2016; Dasari et al., 2019; Peters et al., 2010; Parmas et al., 2019; Pinto & Gupta, 2016; Nachum…”
- Learned Belief Search: Efficiently Improving Policies in Partially Observable Settings (2021, background): “More recently, belief-based search techniques have been scaled to large games, leading to breakthroughs in poker (Morav ˇ c ´ ık et al., 2017; Brown & Sandholm, 2017; 2019) and the cooperative game Han-abi (Lerer et al., 2020), as well as large improvements in Bridge (Tian et al., 2020).”
- Simplified Action Decoder for Deep Multi-Agent Reinforcement Learning (2019, methodology): “This includes examples such as Go (Silver et al., 2016; 2017; 2018), poker (Brown & Sandholm, 2017; Moravˇcík et al., 2017; Brown & Sandholm, 2019) and chess (Campbell et al., 2002).”
- A survey of multi-agent deep reinforcement learning with communication (2022, neutral): “Many real-world scenarios, such as autonomous driving [1], sensor networks [2], robotics [3] and game-playing [4, 5], can be modeled as multi-agent systems.”
- Joint Policy Search for Multi-agent Collaboration with Imperfect Information (2020, background): “om/facebookresearch/jps. 1 Introduction Deep reinforcement learning has demonstrated strong or even super-human performance in many complex games (e.g., Atari [27], Dota 2 [29], Starcraft [41], Poker [5, 4, 28], Find and Seek [1], Chess [8] and Go [33, 35, 38]). While massive computational resources”
- Learn to Interpret Atari Agents (2018, background): “…that achieve human-level performance in complex games are AlphaStar [52] which plays StarCraft II, FTW [53] which plays the 3D multi-player first-person video game, Quake III Arena, and Pluribus [54] which plays multi-player poker and have defeated human professionals in no-limit Texas hold’em.”
- AI deception: A survey of examples, risks, and potential solutions (2023, neutral): “• Bluffs: Pluribus, a poker-playing model created by Meta, successfully bluffed human players into folding (Brown et al. 2019).”
- Neural Replicator Dynamics (2019, neutral): “havior and potential failure modes. In partially-observable zero-sum games, such as poker, traditional approaches using expert knowledge or search with a perfect model have scaled to very large games [8, 11, 12, 51]. However, there have been RL-inspired advances as well: for example Regression Count”
- Harms from Increasingly Agentic Algorithmic Systems (2023, background): “…performance on a wider variety of narrow tasks with limited to no human supervision, including but not limited to increasingly complex board games [Brown and Sandholm 2019; Perolat et al. 2022; Silver et al. 2016, 2017] and video games [Mnih et al. 2013; Schrittwieser et al. 2020; Ye et al. 2021].”
- Measuring the Occupational Impact of AI: Tasks, Cognitive Abilities and AI Benchmarks (2020, background): “For instance, performing well in games such as checkers, chess, Go and poker (Silver et al., 2016, 2017; Brown and Sandholm, 2019), which is recorded in corresponding benchmarks, is not a required ability in any work-related task.”
- Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence (2023, background): “Since then, deep reinforcement learning (RL)- based programs and applications have been developed to determine the degree to which they can compete with human champion players in various games like Texas hold’em poker [11] and Dota 2 [12].”
- Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Making (2021, background): “1 INTRODUCTION As AI performance grows rapidly and often surpasses humans in constrained tasks [4, 23, 27, 48, 57], a critical challenge to enable social good is to understand how AI assistance can be used to enhance human performance.”
- Learning Zero-Sum Simultaneous-Move Markov Games Using Function Approximation and Correlated Equilibrium (2020, background): “…game of Go (Silver et al., 2016, 2017), real-time strategy games (OpenAI, 2018; Vinyals et al., 2019), Texas Hold’em poker (Moravˇcík et al., 2017; Brown and Sandholm, 2018, 2019), autonomous driving (Shalev-Shwartz et al., 2016), and learning communication and emergent behaviors (Foerster et…”
- Rethinking Formal Models of Partially Observable Multiagent Decision Making (2019, methodology): “We start with counterfactual regret minimization, the now-standard algorithm which lies at the heart of recent successes such as [30, 7].”