Point-based value iteration: an anytime algorithm for POMDPs

By Joelle Pineau, Geoffrey J. Gordon, Sebastian Thrun. 2003

1,243 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Hilbert Space Embeddings of POMDPs (2012, methodology): “, 2006, Even-dar, 2005]); Monte Carlo methods for more complex models (Monte Carlo POMDPs [Silver and Veness, 2010, Thrun, 2000]); and methods that take advantage of the piece-wise linear and convex (PWLC) property (PBVI [Pineau et al., 2003], Perseus [Spaan and Vlassis, 2005], HSVI [Smith and Simmo”
Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning (2011, background): “A compact model of a discrete latent space is learned and used for control learning by means of point-based value iteration [16].”
Recurrent Reinforcement Learning: A Hybrid Approach (2015, neutral): “The most common approach to dealing with partially observable states in reinforcement learning is to use a partially observable Markov decision process, or POMDP (Kaelbling et al., 1998), which is found successful in a few domains (Pineau et al., 2003; Williams & Young, 2007).”
Deep Variational Reinforcement Learning for POMDPs (2018, background): “…POMDP literature focusses on planning algorithms, where the transition and observation functions, as well as a representation of the latent state space, are known (Barto et al., 1995; McAllester & Singh, 1999; Pineau et al., 2003; Ross et al., 2008; Oliehoek et al., 2008; Roijers et al., 2015).”
Information Gain-based Exploration Using Rao-Blackwellized Particle Filters (2005, background): “As can be seen from this experiment, as soon as the robot gets too uncertain about its pose, it performs place re-visiting actions or chooses exploration actions which also reduce its pose uncertainty due to the information gathered along the path.”
Planning under Uncertainty for Robotic Tasks with Mixed Observability (2010, methodology): “We would also like to point out that the belief update and backup operations, which are the main modifications required for SARSOP to solve MOMDPs, are common to most pointbased POMDP algorithms, such as PBVI (Pineau et al. 2003a), Perseus (Spaan and Vlassis 2004), and HSVI (Smith and Simmons 2004).”
Partially observable Markov decision processes for spoken dialog systems (2007, unknown): “To illustrate how this POMDP framework is used in a spoken dialog system, an example will now be presented in some detail.”
Stick-Breaking Policy Learning in Dec-POMDPs (2015, methodology): “Specifically, the learning agent is allowed access to episodes collected by taking actions according to a POMDP algorithm (point-based value iteration (PBVI) [Pineau et al., 2003]).”
Linear Fitted-Q Iteration with Multiple Reward Functions (2012, methodology): “, Pineau et al. 2006, Wang et al. 2006) may be useful, but novel modifications will be needed to use these approximations in our setting. Another class of approximations introduced by Poupart and Boutilier (2002) focuses on compressing the state space of the POMDP.”
Control: A perspective (2014, methodology): “With the advent of powerful computation, the problem of “partially observed Markov decision processes,” (POMDPs) [606], has acquired great attention as a methodology for modeling and solving problems in machine learning and artificial intelligence [531,619,501,641,493,245,596].”
Global A-Optimal Robot Exploration in SLAM (2005, methodology): “Good approximation methods have been described recently to solve many POMDPs approximately [12], [13], but the exploration problem has structure that allows even greater computational efficiency.”
A Decision-Theoretic Model of Assistance (2007, methodology): “An example of this method is point based value iteration (PBVI) (Pineau et al., 2003; Porta et al., 2006) that takes a set of belief points B as input and maintains a set of POMDP α-vectors at each iteration.”
Online Planning Algorithms for POMDPs (2008, methodology): “We present various combinations of online algorithms with various existing offline algorithms, such as QMDP (Littman, Cassandra, & Kaelbling, 1995), FIB (Hauskrecht, 2000), Blind (Hauskrecht, 2000; Smith & Simmons, 2005) and PBVI (Pineau et al., 2003).”
POPCORN: Partially Observed Prediction COnstrained ReiNforcement Learning (2020, methodology): “For the modest state spaces in our applications (K 100), PBVI is an efficient solver.”
Finding Approximate POMDP solutions Through Belief Compression (2011, unknown): “In particular, the best algorithms to date for finding an approximate value function over the full belief space have not scaled beyond a few hundred states (Pineau et al., 2003a).”
An Introduction to Deep Reinforcement Learning (2018, unknown): “When the full model (T , R and O) are known, methods such as Point-Based Value Iteration (PBVI) algorithm (Pineau et al., 2003) for POMDP planning can be used to solve the problem.”
Reinforcement Learning with Limited Reinforcement : Using Bayes Risk for Active Learning in POMDPs Finale (2012, methodology): “For planning and b elief monitoring in standard discrete POMDPs, we refer the reader to classic wor k such as [27].”