Better Computer Go Player with Neural Network and Long-term Prediction

By Yuandong Tian, Yan Zhu. 2015

86 citations recorded by the source corpus.

Competing with top human players in the ancient game of Go has been a long-term goal of artificial intelligence. Go's high branching factor makes traditional search techniques ineffective, even on leading-edge hardware, and Go's evaluation function could change drastically with one stone change. Recent works [Maddison et al. (2015); Clark & Storkey (2015)] show that search is not strictly necessary for machine Go players. A pure pattern-matching approach, based on a Deep Convolutional Neural Network (DCNN) that predicts the next move, can perform as well as Monte Carlo Tree Search (MCTS)-based open source Go engines such as Pachi [Baudis & Gailly (2012)] if its search budget is limited. We extend this idea in our bot named darkforest, which relies on a DCNN designed for long-term predictions. Darkforest substantially improves the win rate for pattern-matching approaches against MCTS-based approaches, even with looser search budgets. Against human players, the newest versions, darkfores2, achieve a stable 3d level on KGS Go Server as a ranked bot, a substantial improvement upon the estimated 4k-5k ranks for DCNN reported in Clark & Storkey (2015) based on games against other machine players. Adding MCTS to darkfores2 creates a much stronger player named darkfmcts3: with 5000 rollouts, it beats Pachi with 10k rollouts in all 250 games; with 75k rollouts it achieves a stable 5d level in KGS server, on par with state-of-the-art Go AIs (e.g., Zen, DolBaram, CrazyStone) except for AlphaGo [Silver et al. (2016)]; with 110k rollouts, it won the 3rd place in January KGS Go Tournament.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Economics and Econometrics, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Economics and Econometrics

In Reinforcement Learning

Citations in the corpus

ELF OpenGo: An Analysis and Open Reimplementation of AlphaZero (2019, methodology): “We now describe the system and software design of ELF OpenGo, which builds upon the DarkForest Go engine (Tian & Zhu, 2015) and the ELF reinforcement learning platform (Tian et al., 2017).”
Joint Policy Search for Multi-agent Collaboration with Imperfect Information (2020, methodology): “For perfect information game (PG), α-β pruning, Iterative Deepening depth-first Search [21], Monte Carlo Tree Search [13] are used in Chess [9] and Go [34, 40], yielding strong performances.”
Building Machines that Learn and Think Like People (2018, unknown): “Adapted from Tian and Zhu (2015).”
Can Neural Networks Understand Logical Entailment? (2018, methodology): “The fourth and ﬁfth encoding benchmarks are (tree) recursive neural networks (Tai et al., 2015; Le & Zuidema, 2015; Zhu et al., 2015; Allamanis et al., 2016), also known as TreeRNNs.”
ELF: An Extensive, Lightweight and Flexible Research Platform for Real-time Strategy Games (2017, background): “We can also incorporate forwardplanning methods (e.g., Monte-Carlo Tree Search (MCTS) [7, 30, 25]) and Self-Play, in which at one time step, a single environment might emit multiple states processed by multiple models, or one-to-many.”