Monte-Carlo Planning in Large POMDPs

By David Silver, Joel Veness. 2010

1,335 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Hilbert Space Embeddings of POMDPs (2012, methodology): “…models of the system (parametric POMDPs [Poupart et al., 2006, Even-dar, 2005]); Monte Carlo methods for more complex models (Monte Carlo POMDPs [Silver and Veness, 2010, Thrun, 2000]); and methods that take advantage of the piece-wise linear and convex (PWLC) property (PBVI [Pineau et al.,…”
Learned Belief Search: Efficiently Improving Policies in Partially Observable Settings (2021, methodology): “Silver & Veness (2010) propose a method for performing Monte Carlo Tree Search in large POMDPs like Battleship and partially-observable PacMan.”
Deep Variational Reinforcement Learning for POMDPs (2018, methodology): “For DVRL , we extend the latent state to be a set of K particles, capturing the uncertainty in the belief state (Thrun, 2000; Silver & Veness, 2010).”
The Dependence of Effective Planning Horizon on Model Accuracy (2015, methodology): “We used a benchmark POMDP domain RockSample [14] and evaluated UCT’s performance with different numbers of trajectories and different maximum depths.”
Solving Hidden-Semi-Markov-Mode Markov Decision Problems (2014, methodology): “We chose to base our solving method on POMCP, because it tends to be more efficient than specialized algorithms on MOMDPs and more generally on factored POMDPs, even when POMCP is run on the non-factored representations [16].”
Robust Opponent Modeling via Adversarial Ensemble Reinforcement Learning in Asymmetric Imperfect-Information Games (2019, background): “Most works deal with domains where the hidden state has a well-modeled probabilistic relationship with the observations, such as partial observability due to sensor noise or failure [24], limited field of view [28], screen flickering [9].”
A Bayesian Approach for Learning and Planning in Partially Observable Markov Decision Processes (2011, result): “Recent results in Monte-Carlo Planning provide a good basis for this (Silver and Veness, 2010; Veness et al., 2011).”
DESPOT: Online POMDP Planning with Regularization (2013, methodology): “For POMCP, we used the implementation in [18] but ran it on the same platform as AR-DESPOT.”
On Value Functions and the Agent-Environment Boundary (2019, background): “One can also attempt to reproduce the sequence of observations and actions from the beginning of the episode [see POMCP; Silver and Veness, 2010].”
Reinforcement learning with limited reinforcement: using Bayes risk for active learning in POMDPs (2008, unknown): “In general, the integral over model space is intractable; even new online planning methods [28] cannot achieve the search-depths necessary for pla nning in the modeluncertainty POMDP.”
A survey of uncertainty in deep neural networks (2021, methodology): “In case of planning and control, estimation problems are widely treated as Bayesian sequential learning problems, and sequential decision making frameworks such as POMDPs [310], [311] assume a probabilistic treatment of the underlying planning problems.”
A Survey of the Seventh International Planning Competition (2012, methodology): “For the MDP track, the winner was PROST (team members Thomas Keller and Patrick Eyerich ), which used UCT in combination with determinization techniques to initialize heuristics; the runner up was Glutton (team members Andrey Kolobov, Peng Dai, Mausam, and Daniel Weld), which used an iterative deepe”
Monte Carlo Planning Method Estimates Planning Horizons during Interactive Social Exchange (2015, methodology): “The reason for this is that the probabilities of getting to the nodes in the search tree represent exactly what is necessary to compensate for the apparent inferential infelicity [31]– i.”
Reset-free Trial-and-Error Learning for Robot Damage Recovery (2016, methodology): “exploited by probabilistic planners. Here we use Monte Carlo Tree Search (MCTS) [32], as it has already been succesfully used to solve (Partially Observable)-MDPs with stochastic transition functions [33], [34], continuous state spaces, and high branching factors [34], [35]. MCTS is a best-ﬁrst, sam”
A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity (2017, background): “To solve large-sized HS3MDPs, Hadoux et al. proposed an adaptation of POMCP (Silver and Veness, 2010).”