Online Monte Carlo Counterfactual Regret Minimization for Search in Imperfect Information Games
By Viliam Lisý, Marc Lanctot, Michael H. Bowling. 2015
82 citations recorded by the source corpus.
Online search in games has been a core interest of artificial intelligence. Search in imperfect information games (e.g., Poker, Bridge, Skat) is particularly challenging due to the complexities introduced by hidden information. In this paper, we present Online Outcome Sampling, an online search variant of Monte Carlo Counterfactual Regret Minimization, which preserves its convergence to Nash equilibrium. We show that OOS can overcome the problem of non-locality encountered by previous search algorithms and perform well against its worst-case opponents. We show that exploitability of the strategies played by OOS decreases as the amount of search time increases, and that preexisting Information Set Monte Carlo tree search (ISMCTS) can get more exploitable over time. In head-to-head play, OOS outperforms ISMCTS in games where non-locality plays a significant role, given a sufficient computation time per move.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Johannes Heinrich
- David Silver
- Yuxi Li
- Sergey Levine
- Kaiqing Zhang
- R´emi Munos
- Marc Lanctot
- Pieter Abbeel
Citations in the corpus
- Algorithms for computing strategies in two-player simultaneous move games (2016, unknown): “This paper presents detailed descriptions and analysis of recent stateof-the-art exact [10] and approximative algorithms [11, 12, 13] that compute strategies for the class of two-player simultaneous move games.”
- Smooth UCT Search in Computer Poker (2015, background): “Smooth UCT, on the other hand, continued to approach a Nash equilibrium, but was eventually overtaken by Outcome Sampling.”