Superhuman AI for heads-up no-limit poker: Libratus beats top professionals

By Noam Brown, Tuomas Sandholm. 2018

831 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Economics and Econometrics, Reinforcement Learning, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Economics and Econometrics

In Reinforcement Learning

In Game & Decision Theory

Citations in the corpus

Learned Belief Search: Efficiently Improving Policies in Partially Observable Settings (2021, background): “More recently, belief-based search techniques have been scaled to large games, leading to breakthroughs in poker (Morav ˇ c ´ ık et al., 2017; Brown & Sandholm, 2017; 2019) and the cooperative game Han-abi (Lerer et al., 2020), as well as large improvements in Bridge (Tian et al., 2020).”
Simplified Action Decoder for Deep Multi-Agent Reinforcement Learning (2019, methodology): “This includes examples such as Go (Silver et al., 2016; 2017; 2018), poker (Brown & Sandholm, 2017; Moravˇcík et al., 2017; Brown & Sandholm, 2019) and chess (Campbell et al., 2002).”
The Hanabi Challenge: A New Frontier for AI Research (2019, background): “Hanabi is different from the adversarial two-player zero-sum games where computers have reached super-human skill, e.g. , chess [9], checkers [10], go [11], backgammon [12] and two-player poker [13,14].”
Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learning (2018, methodology): “The continual re-solving (nested solving) algorithm used by DeepStack (Morav ˇ c ´ ık et al., 2017) and Libratus (Brown & Sandholm, 2018) for poker also used a belief state space.”
A survey and critique of multiagent deep reinforcement learning (2018, background): “The research interest inMAL has been accompanied by successes in artificial intelligence, first, in single-agent video games [221]; more recently, in two-player games, for example, playingGo [291,293], poker [50,224], and games of two competing teams, e.”
"Other-Play" for Zero-Shot Coordination (2020, background): “If it converges, self-play ﬁnds a Nash equilibrium of the game and yields superhuman AI agents in two-player zero-sum games such as Chess, Go and Poker (Campbell et al., 2002; Silver et al., 2017; Brown & Sandholm, 2018).”
Improving Policies via Search in Cooperative Partially Observable Games (2019, background): “The most prominent partially observable benchmark game is poker, where search based on beliefs was key to achieving professional-level performance (Moravčı́k et al. 2017; Brown and Sandholm 2017; Brown and Sandholm 2019).”
Depth-Limited Solving for Imperfect-Information Games (2018, background): “The primary benchmark for this class of games is poker, speciﬁcally heads-up no-limit Texas hold’em (HUNL), in which Libratus defeated top humans in 2017 [6].”
Trust in AI: progress, challenges, and future directions (2024, neutral): “…transportation (Qayyum et al., 2020; Schwarting et al., 2018; Sligar, 2020), military (Galán et al., 2022; Roessingh et al., 2017), data security (Aljably et al., 2020; Pan et al., 2019), entertainment (Brown and Sandholm, 2018; Mnih et al., 2013; Morav č ík et al., 2017; Silver et al., 2016) etc.”
Actor-Critic Policy Optimization in Partially Observable Multiagent Environments (2018, background): “L G ] 27 N ov 2018 regret minimization (CFR), which is the root of recent advances in no-limit, such as Libratus [16] and DeepStack [61].”
Superhuman AI for multiplayer poker (2019, methodology): “The two-player poker AI Libratus sidestepped this issue by only doing real-time search when the remaining game was short enough that the depth limit would extend to the end of the game ( 6 ).”
Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf (2023, background): “And superhuman performance has been achieved in some incomplete information games, such as heads-up poker (Bowling et al., 2015; Brown and Sandholm, 2018).”
Computing Approximate Equilibria in Sequential Adversarial Games by Exploitability Descent (2019, methodology): “Based on CFR, recent techniques have solved heads-up limit Texas Hold’em [ Bowling et al. , 2015 ] and beat human professionals in no-limit Texas Hold’em [ Morav ˇ c ´ ık et al. , 2017; Brown and Sandholm, 2017 ] .”
Learning Zero-Sum Simultaneous-Move Markov Games Using Function Approximation and Correlated Equilibrium (2020, background): “…game of Go (Silver et al., 2016, 2017), real-time strategy games (OpenAI, 2018; Vinyals et al., 2019), Texas Hold’em poker (Moravˇcík et al., 2017; Brown and Sandholm, 2018, 2019), autonomous driving (Shalev-Shwartz et al., 2016), and learning communication and emergent behaviors (Foerster et…”
The Bottleneck Simulator: A Model-based Deep Reinforcement Learning Approach (2018, background): “Deep reinforcement learning (RL) has recently shown impressive successes across a variety of tasks (Mnih et al., 2013; Tesauro, 1995; Silver et al., 2017; 2016; Brown & Sandholm, 2017; Watter et al., Lillicrap et al., 2015; Schulman et al., 2015; Levine et al., 2016).”