Move Evaluation in Go Using Deep Convolutional Neural Networks

By Chris J. Maddison, Aja Huang, Ilya Sutskever, David Silver. 2014

136 citations recorded by the source corpus.

Abstract: The game of Go is more challenging than other board games, due to the difficulty of constructing a position or move evaluation function. In this paper we investigate whether deep convolutional networks can be used to directly represent and learn this knowledge. We train a large 12-layer convolutional neural network by supervised learning from a database of human professional games. The network correctly predicts the expert move in 55% of positions, equalling the accuracy of a 6 dan human player. When the trained convolutional network was used directly to play games of Go, without any search, it beat the traditional search program GnuGo in 97% of games, and matched the performance of a state-of-the-art Monte-Carlo tree search that simulates a million positions per move.

Subfields: Artificial Intelligence, Developmental and Educational Psychology, Economics and Econometrics, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Developmental and Educational Psychology

In Economics and Econometrics

In Reinforcement Learning

Citations in the corpus

TensorFlow: A system for large-scale machine learning (2016, neutral): “In recent years, machine learning has driven advances in many different fields [3, 5, 23, 24, 30, 27, 40, 45, 48, 50, 55, 68, 69, 73, 76].”
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems (2016, neutral): “com sequence prediction [47], move selection for Go [34], pedestrian detection [2], reinforcement learning [38], and other areas [17, 5].”
Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm (2017, methodology): “Recently, the AlphaGo Zero algorithm achieved superhuman performance in the game of Go, by representing Go knowledge using deep convolutional neural networks (23, 30), trained solely by reinforcement learning from”
Dueling Network Architectures for Deep Reinforcement Learning (2015, background): “15), and model predictive control with embeddings (Watter et al., 2015). Other recent successes include massively parallel frameworks (Nair et al., 2015) and expert move prediction in the game of Go (Maddison et al., 2015), which produced policies matching those of Monte Carlo tree search programs, ”
Deep Learning with Differential Privacy (2016, background): “Recent progress in neural networks has led to impressive successes in a wide range of applications, including image classiﬁcation, language representation, move selection for Go, and many more (e.g., [49, 27, 52, 37, 46]).”
Learning Multiagent Communication with Backpropagation (2016, background): “Sev ral recent works have applied th se methods t multi-agent domains, such s Go [16, 24] and Atari games [29], but t ey assume fu l visibility of the e vir nm t and lack comm ni ati n.”
Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networks (2016, background): “In competitive settings, deep learn-† These authors contributed equally to this work. ing for Go (Maddison et al., 2015; Silver et al., 2016) has recently shown success.”
StarCraft II: A New Challenge for Reinforcement Learning (2017, methodology): “The use of supervised data such as replays or human demonstrations has been successful in robotics [2, 24], the game of Go [18, 31], and Atari [10].”
Deep Reinforcement Learning from Self-Play in Imperfect-Information Games (2016, background): “In computer Go, Maddison et al. (2015) and Clark and Storkey (2015) trained deep neural networks from data of expert human play.”
Neural Networks and Deep Learning (2018, unknown): “The use of convolutional neural networks for playing the game of Go has been explored in [73, 307, 481].”
Mastering the game of Go without human knowledge (2017, methodology): “…data, we trained a second neural network (using the same architecture) to predict expert moves in the KGS Server data set; this achieved stateoftheart prediction accuracy compared to pre vious work 12,30–33 (see Extended Data Tables 1 and 2 for current and previous results, respectively).”
Mastering the game of Go with deep neural networks and tree search (2016, background): “Although this may be effective in small neural networks, it actually hurts performance in larger networks, as it prevents the intermediate filters from identifying specific asymmetric patterns 23 .”
Artificial intelligence as structural estimation: Deep Blue, Bonanza, and AlphaGo (2017, background): “The condition (20) does not guarantee the superiority of σ (”
Learning how to Active Learn: A Deep Reinforcement Learning Approach (2017, background): “, 2016) and expert move prediction in the game of Go (Maddison et al., 2015), which produced policies matching those of the Monte Carlo tree search programs, and squarely beaten a professional player when combined with search (Silver et al.”
Autonomous Learning of State Representations for Control: An Emerging Field Aims to Autonomously Learn State Representations for Reinforcement Learning Agents from Their Real-World Sensor Observations (2015, background): “[27, 38]) as well as playing Go from raw visual input [32] and learning attention policies for object recognition [36].”
ELF OpenGo: An Analysis and Open Reimplementation of AlphaZero (2019, background): “Go’s square grid game board and the spatial locality of moves naturally suggest the use of convolutional architectures, trained on historical human games (Clark & Storkey, 2015; Maddison et al., 2015; Tian & Zhu, 2015).”