Bayes' Bluff: Opponent Modelling in Poker

By F. Southey, Michael H. Bowling, Bryce Larson, Carmelo Piccione, Neil Burch, Darse Billings, D. C. Rayner. 2005

298 citations recorded by the source corpus.

Poker is a challenging problem for artificial intelligence, with non-deterministic dynamics, partial observability, and the added difficulty of unknown adversaries. Modelling all of the uncertainties in this domain is not an easy task. In this paper we present a Bayesian probabilistic model for a broad class of poker games, separating the uncertainty in the game dynamics from the uncertainty of the opponent's strategy. We then describe approaches to two key sub-problems: (i) inferring a posterior over opponent strategies given a prior distribution and observations of their play, and (ii) playing an appropriate response to that distribution. We demonstrate the overall approach on a reduced version of poker using Dirichlet priors and then on the full game of Texas hold'em using a more informed prior. We demonstrate methods for playing effective responses to the opponent, based on the posterior.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Economics and Econometrics, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Economics and Econometrics

In Game & Decision Theory

Citations in the corpus

Opponent Modeling in Deep Reinforcement Learning (2016, background): “, 1998b; Richards & Amir, 2007), or domain-specific strategies (Schadd et al., 2007; Southey et al., 2005).”
Time and Space: Why Imperfect Information Games are Hard (2018, methodology): “In addition to HULHE, I use Rhode Island hold’em [79], Leduc hold’em [82], and Kuhn poker [56], three synthetic poker games created for artificial intelligence research.”
Reasoning about Hypothetical Agent Behaviours and their Parameters (2017, methodology): “Moreover, our methods can deal with any parameterisation, while the methods proposed in [15,36] are specific to parameters of the used distributions (e.”
A Generalised Method for Empirical Game Theoretic Analysis (2018, methodology): “One application of PSRO was applied to a commonly-used benchmark problem known as Leduc poker [11], except with a fixed action space and penalties for taking illegal moves.”
Neural Replicator Dynamics (2019, background): “In Leduc Poker [56], players instead have limitless chips, one initial private card, and ante 1 chip to play.”
Computing Approximate Equilibria in Sequential Adversarial Games by Exploitability Descent (2019, methodology): “Our experiments are run across four different imperfect information games (see [ Kuhn, 1950 ] , [ Southey et al. , 2005 ] , and [ Lanctot, 2013, Chapter 3 ] ).”
Using counterfactual regret minimization to create competitive multiplayer poker agents (2010, methodology): “Therefore, a small heads-up game, called Leduc Hold’em, which includes all of these important properties, was created [ 16 ].”
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms (2019, methodology): “Various methods have been shown successful for simple variations of poker such as Kuhn poker (Kuhn, 1950) and Leduc hold’em (Southey et al., 2005).”
Dynamic Thresholding and Pruning for Regret Minimization (2017, methodology): “Leduc Hold’em is a popular benchmark for imperfect-information game solving due to its feasible size and strategic complexity.”
Solving Games with Functional Regret Estimation (2014, methodology): “Leduc Hold’em is a poker game based on Kuhn poker (Southey et al. 2005).”