Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search

By Rémi Coulom. 2006

1,768 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Economics and Econometrics, Reinforcement Learning, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Economics and Econometrics

In Reinforcement Learning

In Game & Decision Theory

Citations in the corpus

AI safety via debate (2018, methodology): “The MNIST debate game is simple enough that we can play it with pure Monte Carlo Tree Search [Coulom, 2006] without training a heuristic as in Silver et al.”
Mastering Atari, Go, chess and shogi by planning with a learned model (2019, methodology): “Once a model has been constructed, it is straightforward to apply MDP planning algorithms, such as value iteration [31] or Monte-Carlo tree search (MCTS) [7], to compute the optimal value or optimal policy for the MDP.”
OpenSpiel: A Framework for Reinforcement Learning in Games (2019, methodology): “Search Algorithms There are two classical search algorithms for zero-sum turn-taking games of perfect information: minimax (and alpha-beta) search [29, 59], and Monte Carlo tree search (MCTS) [18, 30, 16].”
TreeQN and ATreeC: Differentiable Tree Planning for Deep Reinforcement Learning (2017, background): “Traditionally, such methods have been limited to domains where perfect environment simulators are available, such as board or card games (Coulom, 2006; Sturtevant, 2008).”
Structured agents for physical construction (2019, methodology): “Our family of deep reinforcement learning (RL) agents can support: (1) vector, sequence, image, and graph-structured representations of scenes; (2) continuous and discrete actions, in absolute or object-centric coordinates; (3) model-free learning via deep Q-learning (Mnih et al., 2015), or actor-cr”
Imagination-Augmented Agents for Deep Reinforcement Learning (2017, methodology): “We compare our regular I2A agent to a variant of Monte Carlo Tree Search (MCTS), which is a modern guided tree search algorithm [12, 26].”
Move Evaluation in Go Using Deep Convolutional Neural Networks (2014, background): “…quantity and quality of prior knowledge, which is used to bias the search towards more promising states in both the search tree and during rollouts (Coulom, 2007; Gelly & Silver, 2011; Enzenberger et al., 2010; Huang et al., 2011), and it is widely believed that this knowledge is the major…”
End-to-end Differentiable Proving (2017, methodology): “To overcome the computational limitations of the end-to-end differentiable prover introduced in this paper, we want to investigate the use of hierarchical attention [25] and reinforcement learning methods such as Monte Carlo tree search [60, 61] that have been used for learning to play Go [62] and c”
Interactive Fiction Games: A Colossal Adventure (2019, positive): “for reinforcement learning agents, but also supports the ability to load and save game states, enabling planning algorithms Monte-Carlo Tree Search [Coulom, 2007] as well as reinforcement learning approaches that rely on the ability to restore state such as Backplay [Resnick et al.”
Learning model-based planning from scratch (2017, background): “, A*, beam search, Monte Carlo Tree Search [5]).”
From Language to Programs: Bridging Reinforcement Learning and Maximum Marginal Likelihood (2017, background): “The RL answer would be better exploration, which can take many forms including simple action-dithering such as -greedy, entropy regularization (Williams and Peng, 1991), Monte Carlo tree search (Coulom, 2006), randomized value functions (Osband et al.”
Learning to Search with MCTSnets (2018, methodology): “One particularly powerful and general method for planning is Monte-Carlo tree search (MCTS) (Coulom, 2006; Kocsis & Szepesvári, 2006), as used in the recent AlphaGo program (Silver et al.”
COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics (2022, positive): “We also report the results of TSMH [57] as another recent baseline which uses Monte-Carlo Tree Search [5].”
Self-play Monte-Carlo tree search in computer poker (2014, background): “MCTS (Coulom 2006) is a simulation-based search algorithm.”
Monte Carlo Tree Search: a review of recent modifications and applications (2021, methodology): “Ikeda and Viennot (2013a) involve Bradley-Terry Model of static knowledge (Coulom, 2007) to bias UCB. Bias in MCTS can affect the performance in terms of suboptimal moves (Imagawa and Kaneko, 2015). MCTS has been also employed to Dynamic Difficulty Adjustment (DDA) (Hunicke, 2005). DDA is a family o”
Static and Dynamic Values of Computation in MCTS (2020, methodology): “Monte Carlo tree search (MCTS) is a widely used approximate planning method that has been successfully applied to many challenging domains such as computer Go [4, 21].”
Prioritized memory access explains planning and hippocampal replay (2017, neutral): “Both of these patterns (forward and reverse value propagation) have precedent in different computer science methods (e.g. Monte Carlo tree search [52] and Prioritized Sweeping [27]).”
Reasoning with Language Model is Planning with World Model (2023, methodology): “We do 20 iterations for MCTS and 20 samples for self-consistency.”