OpenSpiel: A Framework for Reinforcement Learning in Games

By Marc Lanctot, Edward Lockhart, Jean-Baptiste Lespiau, Vinícius Flores Zambaldi, S. Upadhyay, Julien Pérolat, Sriram Srinivasan, Finbarr Timbers, Karl Tuyls, Shayegan Omidshafiei and 16 more. 2019

323 citations recorded by the source corpus.

OpenSpiel is a collection of environments and algorithms for research in general reinforcement learning and search/planning in games. OpenSpiel supports n-player (single- and multi- agent) zero-sum, cooperative and general-sum, one-shot and sequential, strictly turn-taking and simultaneous-move, perfect and imperfect information games, as well as traditional multiagent environments such as (partially- and fully- observable) grid worlds and social dilemmas. OpenSpiel also includes tools to analyze learning dynamics and other common evaluation metrics. This document serves both as an overview of the code base and an introduction to the terminology, core concepts, and algorithms across the fields of reinforcement learning, computational game theory, and search.

Subfields: Reinforcement Learning, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Game & Decision Theory

Citations in the corpus

From Poincaré Recurrence to Convergence in Imperfect Information Games: Finding Equilibrium via Regularization (2020, methodology): “, 2018) to Follow the Regularized Leader in monotone Imperfect Information Games when the reward is policyindependent (all the zero-sum two-player games implemented in OpenSpiel (Lanctot et al., 2019) have this property) and when the equilibrium has a full support.”
Monte Carlo Tree Search With Iteratively Refining State Abstractions (2021, methodology): “We modiﬁed OpenSpiel’s [17] AlphaZero and Go implementations for this experiment.”
Physically Embedded Planning Problems: New Challenges for Reinforcement Learning (2020, methodology): “To handle the underlying abstract game logic of tic-tac-toe and Go, we leverage the OpenSpiel framework (Lanctot et al., 2019).”
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms (2019, background): “See [114] for a more detailed discussion on the connection.”
Neural Replicator Dynamics (2019, unknown): “ant with 5 cards where bid cards are not revealed [35]. The number of information states for the Kuhn, Leduc, and Goofspiel variants evaluated is 12, 936, and 2124, respectively; we use the OpenSpiel [37] implementations of these three games in our experiments. We first show that the differences bet”
PettingZoo: Gym for Multi-Agent Reinforcement Learning (2020, unknown): “…learning libraries or code bases exist (as a rough measure 669 pip-installable packages depend on it at the time of writing GitHub [2021]), only 5 MARL libraries with large user bases exist [Lanctot et al., 2019, Weng et al., 2020, Liang et al., 2018, Samvelyan et al., 2019, Nota, 2020].”
A review of cooperative multi-agent deep reinforcement learning (2019, unknown): “StarCraft II [166], Particle [140], Unity [13, 89, 99, 148],”