Temporal Difference Learning of Position Evaluation in the Game of Go

By N. Schraudolph, Peter S. Dayan, Terrence J. Sejnowski. 1993

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Giraffe: Using Deep Reinforcement Learning to Play Chess (2015, methodology): “The same approach was used on Go by Schraudolph et al. [20], but they were only able to achieve ”a low playing level” [20], due to the complexity of the game, and the necessity to look ahead.”
Move Evaluation in Go Using Deep Convolutional Neural Networks (2014, methodology): “Although CNNs have previously been applied to the game of Go, with modest success (Schraudolph et al., 1994; Enzenberger, 1996; Sutskever & Nair, 2008), previous architectures have typically been limited to one hidden layer of relatively small size, and have not exploited recent advances in…”
Deep Successor Reinforcement Learning (2016, methodology): “In particular, the SR can adapt quickly to changes in distal reward, unlike model-free algorithms.”
Better Computer Go Player with Neural Network and Long-term Prediction (2015, methodology): “Using Neural Network as a function approximator and pattern matcher to predict the next move of Go is a long-standing idea [Sutskever & Nair (2008); Richards et al. (1998); Schraudolph et al. (1994); Enzenberger (1996)].”
Reinforcement learning in the game of Othello: Learning against a fixed opponent and learning from self-play (2013, background): “Other RL applications to games include chess [4], checkers [5] and Go [6].”
Learning to Play Chess Using Temporal Differences (2000, background): “Many authors have discussed the peculiarities of backgammon that make it particularly suitable for Temporal Difference learning with self-play (Tesauro, 1992; Schraudolph et al., 1994; Pollack, Blair, & Land, 1996).”
An Introduction to Deep Reinforcement Learning (2018, unknown): “Board-games have also been used for evaluating artificial intelligence methods for decades (Shannon, 1950; Turing, 1953; Samuel, 1959; Sutton, 1988; Littman, 1994; Schraudolph et al., 1994; Tesauro, 1995; Campbell et al., 2002).”
KnightCap: A chess program that learns by combining TD(lambda) with game-tree search (1999, background): “Many authors have discussed the peculiarities of backgam-mon that make it particularly suitable for Temporal Difference learning with self-play [8, 6, 4].”
Mastering the game of Go without human knowledge (2017, methodology): “This neural network was trained by temporaldifference learning 42 to predict territory in games of selfplay, building on previous work 43 .”
On the role of tracking in stationary environments (2007, background): “In a complex domain such as Go, it is usual to seek the best approximation to the optimal policy that can be achieved by a particular representation, for example a linear combination of binary features (Silver, Sutton & M¨uller, 2007), or a multi-layer perceptron ( Schraudolph, Dayan & Sejnowski, 19”
Coevolution of a Backgammon Player (1996, methodology): “Schraudolph et al., 1994 added non-determinism to the game of Go by choosing moves according to the Boltzmann distribution of statistical mechanics.”
Learning to play Go using recursive neural networks (2008, background): “Pure bootstrap approaches to Go where computer players are initialized randomly and play large numbers of games, such as evolutionary approaches or reinforcement learning (Sutton and Barto, 1998), have been tried (Schrauldolph et al., 1994).”
Combining online and offline knowledge in UCT (2007, methodology): “Other value-based reinforcement learning methods o.er a rich variety of function approximation methods for state abstraction in complex domains (Schraudolph et al., 1994; Enzenberger, 2003; Sutton, 1996).”
Reinforcement Learning of Local Shape in the Game of Go (2007, methodology): “We use weight sharing to exploit several symmetries of the Go board [8].”
Sample-based learning and search with permanent and transient memories (2008, background): “In the domain of Computer Go, the most successful learning methods have used sample-based reinforcement learning to extract domain knowledge from games of self-play (Schraudolph et al., 1994; Dahl, 1999; Enzenberger, 2003; Silver et al., 2007).”