Temporal-difference search in computer Go

By David Silver, Richard S. Sutton, Martin Müller. 2012

103 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Economics and Econometrics, Sociology and Political Science, Reinforcement Learning, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Economics and Econometrics

In Sociology and Political Science

In Reinforcement Learning

In Game & Decision Theory

Citations in the corpus

TreeQN and ATreeC: Differentiable Tree Planning for Deep Reinforcement Learning (2017, methodology): “, 2017) take a similar approach but use the model to construct a look-ahead tree only when constructing bootstrap targets and selecting actions, similarly to TD-search (Silver et al., 2012).”
Value Prediction Network (2017, methodology): “In order to train a VPN, we propose a combination of temporal-difference search [28] (TD search) and n-step Q-learning [20].”
Distributed Policy Evaluation Under Multiple Behavior Strategies (2013, background): “The feature representation is convenient in problems with very large state dimensions [2] since, even if the state of the system can be known precisely, it is computationally more efficient to work with features with smaller dimension than the size of the original state-space.”
Mastering the game of Go without human knowledge (2017, methodology): “Both NeuroGo and RLGO achieved a weak amateur level of play.”
Bayesian Learning of Recursively Factored Environments (2013, background): “Having access to a model also allows for hybrid techniques such as Dyna (Sutton, 1991; Silver et al., 2008) and TD-Search (Silver et al., 2012), which use a model of the environment to enhance the Proceedings of the 30 th International Conference on Machine Learning, Atlanta, Georgia, USA, 2013.”
Self-play Monte-Carlo tree search in computer poker (2014, methodology): “For example, function approximation and bootstrapping are two ideas that can dramatically improve the efficiency of learning algorithms (Tesauro 1992; Silver, Sutton, and Müller 2012).”
Value-gradient learning (2012, methodology): “RL literature; one example is computer-go using the “TD-Search” algorithm (Silver et al., 2012).”
Bayes-Adaptive Simulation-based Search with Value Function Approximation (2014, methodology): “This approach was originally suggested in the context of planning in large MDPs[19]; we extend it to the case of Bayes-Adaptive planning.”
Scalable Planning and Learning for Multiagent POMDPs (2014, methodology): “s, so they do not address the large observation space in MPOMDPs. The factorization of the history in FTs is not unlike the use of linear function approximation for the state components in TD-Search (Silver, Sutton, and Muller, 2012). However, in contrast to that method, due to our¨ particular facto”
Deep Reinforcement Learning (2018, methodology): “Value prediction networks (VPNs, Oh et al., 2017) take a similar approach but use the model to construct a look-ahead tree only when constructing bootstrap targets and selecting actions, similarly to TD-search (Silver et al., 2012).”
Monte Carlo Tree Search: a review of recent modifications and applications (2021, methodology): “Subsequently, in 2016, AlphaGo won 4–1 in a match versus Lee Sedol (Silver et al.”