Monte Carlo POMDPs

By Sebastian Thrun. 1999

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep Variational Reinforcement Learning for POMDPs (2018, methodology): “For DVRL , we extend the latent state to be a set of K particles, capturing the uncertainty in the belief state (Thrun, 2000; Silver & Veness, 2010).”
An Expectation Maximization Algorithm for Continuous Markov Decision Processes with Arbitrary Reward (2009, background): “These include direct simulation and approximate dynamic programming; see for example [2, 12, 18].”
Dynamic Programming for Structured Continuous Markov Decision Problems (2004, methodology): “…continuous state variables in MDPs are to use function approximators (Gordon 1995; Boyan & Moore 1995), such as artificial neural networks, to discretize the continuous state space more or less naively, which does not scale well to multiple dimensions, or to use Monte-Carlo approaches (Thrun 2000).”
Spoken Dialogue Management Using Probabilistic Reasoning (2000, methodology): “There are several POMDP algorithms that may be the natural choice for policy generation (Sondik, 1971; Monahan, 1982; Parr and Russell, 1995; Cassandra et al., 1997; Kaelbling et al., 1998; Thrun, 1999).”
An Introduction to MCMC for Machine Learning (2004, methodology): “These algorithms have been shown to work with large dimensional models such as neural networks (Neal, 1996) and Gaussian processes ( Barber & Williams, 1997 ).”
Decentralized planning under uncertainty for teams of communicating agents (2006, background): “The “Multiagent Heaven or Hell” problem (inspired by [26]) is a scenario in which two agents have to meet in one of two locations, one called “heaven” and the other one “hell”, see Fig.”
Control: A perspective (2014, methodology): “With the advent of powerful computation, the problem of “partially observed Markov decision processes,” (POMDPs) [606], has acquired great attention as a methodology for modeling and solving problems in machine learning and artificial intelligence [531,619,501,641,493,245,596].”
Bayes-Adaptive Simulation-based Search with Value Function Approximation (2014, background): “In the POMDP literature, a key idea to represent beliefs is to sample a finite set of (possibly approximate) belief points [21, 16] from the set of possible beliefs in order to obtain a small number of (belief-)states for which to backup values offline or via forward search [13].”
Finding Approximate POMDP solutions Through Belief Compression (2011, unknown): “It is interesting to note that this policy contains a similar “coastal” attribute as some heuristic policies (e.g., the Entropy heuristic and the AMDP, Cassandra, Kaelbling, & Kurien, 1996; Roy & Thrun, 1999).”
Trajectory Optimization using Reinforcement Learning for Map Exploration (2008, methodology): “Although there has been work in solving continuous POMDPs (Thrun 2000 Porta et al. 2006), the difficulty in scaling these algorithms makes them ineffective in addressing the exploration problem.”
Reinforcement learning for factored Markov decision processes (2002, methodology): “Recent research has investigated subsets of these issues both theoretically and empirically [McAllester and Singh 1999; Poupart and Boutilier 2000; Thrun 2000; Rodriguez, Parr, and Koller 2000; Sallans 2000].”
Reinforcement Learning with Factored States and Actions (2004, methodology): “The result has been new algorithms and methods for learning about decision processes and making decisions under uncertainty in complex and noisy environments (see for example Boutilier and Poole, 1996; McAllester and Singh, 1999; Thrun, 2000; Sallans, 2000; Rodriguez et al., 2000; Poupart and Boutil”
Learning Deep Policies for Robot Bin Picking by Simulating Robust Grasping Sequences (2017, background): “1) specifed as a tuple (X ,U ,Y, R, ρ0, p, q) consisting of a set of states X (object shapes and poses), a set of actions U (gripper poses), a set of observations Y (point clouds), a reward function R, an initial state distribution ρ0 (object heaps), a next state distribution p, and a sensor noise d”
Particle methods for change detection, system identification, and control (2004, methodology): “Thrun develops a nearest-neighbor type method to perform quantization in this space [55].”
Sampling-based robotic information gathering algorithms (2014, methodology): “Sampling-based approaches have been applied to POMDPs in the past (Thrun, 1999).”
Bayesian Reinforcement Learning in Factored POMDPs (2018, background): “More common approaches approximate the belief with particle filters [44].”
A POMDP formulation of preference elicitation problems (2002, methodology): “Particle filter models can also be used to solve continuous state POMDPs [15].”