Reinforcement Learning of Local Shape in the Game of Go

By David Silver, Richard S. Sutton, Martin Müller. 2007

150 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Language Understanding for Text-based Games using Deep Reinforcement Learning (2015, methodology): “Q-Learning Reinforcement Learning is a commonly used framework for learning control policies in game environments (Silver et al., 2007; Amato and Shani, 2010; Branavan et al., 2011b; Szita, 2012).”
Fast Gradient-Descent Methods for Temporal-Difference Learning with Linear Function Approximation (2009, unknown): “In applications, n is often too large (e.g., n = 10 6 in Computer Go, Silver et al. 2007) for second-order methods to be feasible.”
Sample-based learning and search with permanent and transient memories (2008, methodology): “Similarly, in the permanent memory of our architecture, the value function is approximated by a linear combination of binary features, learned offline by temporal-difference learning and self-play (Silver et al., 2007).”
Value-Aware Loss Function for Model-based Reinforcement Learning (2017, background): “The features φ might either be manually-designed (e.g., [Silver et al., 2007; Liang et al., 2016]) or automatically generated data-dependently, e.g., [Petrik, 2007; Mahadevan and Maggioni, 2007; Parr et al., 2007; Mahadevan and Liu, 2010; Geramifard et al., 2011; Farahmand and Precup, 2012; B¨ohmer…”
Algorithms for Reinforcement Learning (2010, background): “, Backgammon (Tesauro, 1994) and Go (Silver et al., 2007)), applications in networking (e.”
Combining online and offline knowledge in UCT (2007, methodology): “The .gure shows (clockwise from top-left) 1 × 1, 2 × 1, 2 × 2 and 3 × 3 local shap e features which occur in the example position. f(s, a) that recognise local patterns of stones (Silver et al., 2007).”
Temporal-difference search in computer Go (2012, unknown): “3 we used TD learning to extract general domain knowledge from games of self-play (Schraudolph et al. 1994; Enzenberger 1996; Dahl 1999; Enzenberger 2003; Silver et al. 2007).”
Unsupervised Real-Time Control Through Variational Empowerment (2017, unknown): “…Peters, 2012], stock trading [Li et al., 2009, Tsitsik-lis and Van Roy, 1999, 2001], targeted marketing [Abe et al., 2004], learning to play games [Silver et al., 2016, 2007, Tesauro, 1994], predictive maintainance [Gosavi, 2004], process automation [Zhang and Dietterich, 1995] or electrical…”
Regularization in reinforcement learning (2011, methodology): “And finally, reinforcement learning algorithms have also been used to design automated players for games such as backgammon [Tesauro, 1994], Go [Silver et al., 2007], and Atari 2600 console games [Naddaf, 2010].”
A unifying framework for computational reinforcement learning theory (2009, methodology): “many other learning-to-control problems in a broad sense, including the early checkerslearning program [Samuel 1959], the celebrated TD-Gammon program [Tesauro 1995] that achieved world-champion-level game-playing ability in backgammon, packet routing [Boyan and Littman 1994], elevator dispatching [”
Gradient temporal-difference learning algorithms (2011, methodology): “He used a version of RLGO (Silver et al. 2007) modified to use a purely linear evaluation function.”
Fast gradient-descent methods for temporal-difference learning with linear function approximation (2009, unknown): “We used a version of RLGO (Silver et al. 2007) modified to use a purely linear evaluation function.”
Policy evaluation with temporal differences: a survey and comparison (2015, methodology): “For example, they can provide failure probabilities in large telecommunication networks (Frank et al., 2008), taxi-out times at big airports (Balakrishna et al., 2010) or the importance of different board configurations in the game Go (Silver et al., 2007).”
DEVELOPING A PREDICTIVE APPROACH TO KNOWLEDGE (2015, methodology): “In practice, the linear TD(λ) algorithm can outperform the wellknown least mean squares rule (also known as Widrow-Hoff algorithm) on temporal prediction tasks (Sutton, 1988), and has been successfully applied to more complex domains like Backgammon (Tesauro, 1995), Computer GO (Silver et al., 2007)”