ELF OpenGo: An Analysis and Open Reimplementation of AlphaZero

By Yuandong Tian, Jerry Ma, Qucheng Gong, Shubho Sengupta, Zhuoyuan Chen, James Pinkerton, C. Lawrence Zitnick. 2019

120 citations recorded by the source corpus.

The AlphaGo, AlphaGo Zero, and AlphaZero series of algorithms are remarkable demonstrations of deep reinforcement learning's capabilities, achieving superhuman performance in the complex game of Go with progressively increasing autonomy. However, many obstacles remain in the understanding of and usability of these promising approaches by the research community. Toward elucidating unresolved mysteries and facilitating future research, we propose ELF OpenGo, an open-source reimplementation of the AlphaZero algorithm. ELF OpenGo is the first open-source Go AI to convincingly demonstrate superhuman performance with a perfect (20:0) record against global top professionals. We apply ELF OpenGo to conduct extensive ablation studies, and to identify and analyze numerous interesting phenomena in both the model training and in the gameplay inference procedures. Our code, models, selfplay datasets, and auxiliary data are publicly available.

Subfields: Artificial Intelligence, Economics and Econometrics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Economics and Econometrics

Citations in the corpus

Neural Architecture Search Using Deep Neural Networks and Monte Carlo Tree Search (2018, neutral): “AlphaGo/AlphaGoZero (Silver et al. 2016; Tian et al. 2019) have recently shown super-human performance in playing the game of Go, by using a speciﬁc search al-gorithm called Monte-Carlo Tree Search (MCTS) (Kocsis and Szepesv´ari 2006).”
DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames (2019, background): “Several works have proposed systems for distributed RL (Heess et al., 2017; Liang et al., 2018a; Tian et al., 2019; Silver et al., 2016; OpenAI, 2018; Espeholt et al., 2018).”
How to Train PointGoal Navigation Agents on a (Sample and Compute) Budget (2020, background): “Galvanized by fast simulation platforms [15, 20, 21, 29], large rich 3D datasets [4, 25, 30], and the success of deep reinforcement learning [18, 24, 26], training virtual robots (embodied agents) in simulation has garnered considerable interest in recent years.”
The Scientific Method in the Science of Machine Learning (2019, background): “Large scale studies of published work have attempted to provide the community with independent re-implementations of machine learning experiments (Pineau et al., 2017; Tian et al., 2019).”
Supervised Learning Achieves Human-Level Performance in MOBA Games: A Case Study of Honor of Kings (2020, background): “Many other types of Go agents were also proposed, e.g., FineArt (the champion of the UEC Cup in 2017) from Tencent, and OpenGo (open-sourced reimplementation of AlphaGo) from Facebook [21].”
Joint Policy Search for Multi-agent Collaboration with Imperfect Information (2020, background): “, Atari [28], Dota 2 [30], Starcraft [42], Poker [5, 29], Find and Seek [1], Chess, Go and Shogi [34, 36, 39]).”
A Hierarchy of Limitations in Machine Learning (2020, background): “…in doing something correctly can be considered a limitation, and indeed the difficulties of reproducible code and sensitivity to initial conditions in machine learning (Tian et al., 2019) is an increasing source of doubt for the overall reliability of machine learning research (Hutson, 2018).”
Mastering Atari Games with Limited Data (2021, methodology): “To give a better Q-value estimation of the unvisited nodes, we evaluate a mean Q value mechanism in each simulation for tree nodes, similar to the implementation of Elf OpenGo [10].”