Point-Based Value Iteration for Continuous POMDPs

By Josep M. Porta, Nikos A. Vlassis, Matthijs T. J. Spaan, Pascal Poupart. 2006

210 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, background): “24) is contractive and isotonic which guarantee that the value iteration converges monotonically (Porta et al. 2006), the kernel Bellman operator in (4.”
Hilbert Space Embeddings of POMDPs (2012, background): “, the original value iteration is guaranteed to converge monotonically [Porta. et al., 2006], the resulting kernel Bellman operator does not.”
An Expectation Maximization Algorithm for Continuous Markov Decision Processes with Arbitrary Reward (2009, methodology): “One notable exception is the value iteration algorithm proposed in (Porta et al., 2006) for solving partially observed Markov decision processes (POMDPs) with a linear-Gaussian transition model and a mixture of Gaussians reward model.”
Winning Isn't Everything: Training Human-Like Agents for Playtesting and Game AI (2019, background): “While the problem of a huge state space [39], [40], [41], a continuous action space [42], and a parametric actions space [43] could be dealt with, these techniques are not directly applicable to our problem.”
A Decision-Theoretic Model of Assistance (2007, methodology): “An example of this method is point based value iteration (PBVI) (Pineau et al., 2003; Porta et al., 2006) that takes a set of belief points B as input and maintains a set of POMDP α-vectors at each iteration.”
Convex Optimization: Algorithms and Complexity (2015, unknown): “(2.8) The optimal value function for a finite-horizon POMDP can be shown to be piecewise-linear and convex [Smallwood and Sondik, 1973, Porta et al., 2006].”
Bayes-Adaptive Simulation-based Search with Value Function Approximation (2014, unknown): “5 Related Work Simulation-based search with value function approximation has been investigated in large and also continuous MDPs, in combination with TD-learning [19] or Monte-Carlo control [3].”
Trajectory Optimization using Reinforcement Learning for Map Exploration (2008, methodology): “Although there has been work in solving continuous POMDPs (Thrun 2000 Porta et al. 2006), the difficulty in scaling these algorithms makes them ineffective in addressing the exploration problem.”
Reinforcement Learning in Rich-Observation MDPs using Spectral Methods (2016, methodology): “policies (which are sub-optimal in ROMDPs). While the computation of the optimal memoryless policy relies on an optimization oracle, which in general is NP-hard Littman (1994); Vlassis et al. (2012); Porta et al. (2006); Azizzadenesheli et al. (2016a); Shani et al. (2013), computing the optimal poli”