Monte-Carlo tree search and rapid action value estimation in computer Go

By Sylvain Gelly, David Silver. 2011

391 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Economics and Econometrics, Sociology and Political Science

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Economics and Econometrics

In Sociology and Political Science

Citations in the corpus

Move Evaluation in Go Using Deep Convolutional Neural Networks (2014, positive): “…of prior knowledge, which is used to bias the search towards more promising states in both the search tree and during rollouts (Coulom, 2007; Gelly & Silver, 2011; Enzenberger et al., 2010; Huang et al., 2011), and it is widely believed that this knowledge is the major bottleneck towards…”
A Survey of Monte Carlo Tree Search Methods (2012, neutral): “In addition, the use of transpositions allows information to be shared between these extra nodes where the state is unchanged.”
Mastering the game of Go without human knowledge (2017, neutral): “MCTS programs have previously achieved strong amateur level in Go 46,47 , but used substantial domain expertise: a fast rollout policy, based on handcrafted features 13,48 , that evaluates positions by running simulations until the end of the game; and a tree policy, also based on handcrafted…”
Risk-Averse Bayes-Adaptive Reinforcement Learning (2021, positive): “MCTS to two-player games such as Go [14], we use MCTS to handle the large state space by focusing search in promising areas.”
On the Analysis of Complex Backup Strategies in Monte Carlo Tree Search (2016, neutral): “Various modifications have been proposed to improve the exploration strategy under UCT (Gelly and Wang, 2006; Wang and Gelly, 2007) and its value estimation (Gelly and Silver, 2007; 2011; Helmbold and Parker- Wood, 2009; Feldman and Domshlak, 2014b).”
Mastering the game of Go with deep neural networks and tree search (2016, neutral): “AlphaGo does not employ the all-moves-as-first 10 or rapid action value estimation 58 heuristics used in the majority of Monte Carlo Go programs; when using policy networks as prior knowledge, these biased heuristics do not appear to give any additional benefit.”
From Bandits to Monte-Carlo Tree Search: The Optimistic Principle Applied to Optimization and Planning (2014, positive): “Several major improvements (such as the use of features in the random playouts, the Rapid Action Value Estimation (RAVE), the parallelization of the algorithm, and the introduction of opening books) [61, 100, 24, 106, 48, 62] enabled the MoGo program to rank among the best Computer Go programs (see ”
A Survey of Multi-Objective Sequential Decision-Making (2013, positive): “Such approaches, which have proven very successful, e.g., in the game of Go (Gelly & Silver, 2011), are increasingly popular for single-objective MDPs. Wang and Sebag (2013) propose a Monte Carlo tree search method for deterministic MOMDPs.”
Temporal-difference search in computer Go (2012, positive): “In practice, the strongest current Go programs deal with these issues by using domain specific prior knowledge (Chaslot et al. 2008), generalisation among subtrees (Gelly and Silver 2011), and carefully hand-tuned simulation policies (Gelly et al. 2006).”
Monte Carlo Tree Search: a review of recent modifications and applications (2021, positive): “To make more use of simulations, Gelly and Silver (2011) proposes a method called Rapid Value Action Estimation (RAVE).”
Algorithms for computing strategies in two-player simultaneous move games (2016, neutral): “tical success in computer Go [47, 48], MCTS has since been applied successfully to simultaneous move games and to imperfect information games [49, 50, 13].”
The grand challenge of computer Go (2012, positive): “In MoGo, the prior value H(s, a) was originally learned using reinforcement learning and self-play, and N∗(s, a) (and Ñ∗(s, a) in the case of MC–RAVE) was simply set to a constant value [15].”
Artificial intelligence in drug discovery: recent advances and future perspectives (2021, positive): “Inspired by progress in reinforcement learning, one of the most important breakthroughs in the last few years came from the widespread usage of sophisticated search methods, such as Monte Carlo Tree Search [173], to efficiently navigate through chemical reaction spaces [174].”
Online Planning for Large Markov Decision Processes with Hierarchical Decomposition (2015, neutral): “Furthermore, it has been shown to be very useful for exploration in a large solution space [Gelly and Silver 2011].”