The optimal control of par-tially observable Markov processes

By E. Sondik. 1971

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Hilbert Space Embeddings of POMDPs (2012, background): “Experimental results confirm that the correct policy is learned using the feature space representation.”
Concrete Problems for Autonomous Vehicle Safety: Advantages of Bayesian Deep Learning (2017, methodology): “This relies on tools developed in the partially observable Markov Decision processes literature [Astrom, 1965; Sondik, 1971].”
Closing the learning-planning loop with predictive state representations (2009, background): “(POMDPs) [24, 3] are a general framework for singleagent planning.”
Reinforcement Learning of POMDPs using Spectral Methods (2016, methodology): “In this case, it is more appropriate to use the partially-observable MDP or POMDP (Sondik, 1971) model.”
Belief space planning assuming maximum likelihood observations (2010, background): “For a ﬁnite planning horizon, the value function is piecewise linear with a maximum number of facets doubly exponential in the number of observations and the planning horizon [5].”
Acting under uncertainty: discrete Bayesian models for mobile-robot navigation (1996, background): “mdp [19, 3, 11, 9], but even the most e cient of these would do well to solve problems with 10 states and 10 observations.”
Heuristic Search Value Iteration for POMDPs (2004, background): “Even for the discounted inﬁnite-horizon formulation, a ﬁnite vector set can approximate V ∗ arbitrarily closely.”
Joint Design and Separation Principle for Opportunistic Spectrum Access in the Presence of Sensing Errors (2007, methodology): “We can thus use the existing computationally efficient algorithms [4]–[6] to solve (8) for the optimal sensing policy.”
Multi-task Reinforcement Learning in Partially Observable Stochastic Environments (2009, methodology): “Since the state in a POMDP is only partially observable, the action choice is based on the belief state, a sufficient statistic defined as the probability distribution of the state s given the history of actions and observations (Sondik 1971).”
Monte Carlo POMDPs (1999, background): “Initial empirical results suggest that our approach works well in practical applications.”
Modeling of risk-based inspection, maintenance and life-cycle cost with partially observable Markov decision processes (2005, background): “…management, machine maintenance, reservoir control, structural inspection and rehabilitation, and the control of physical systems subject to random disturbances (Bellman 1957, Blackwell 1965, Dynkin 1965, Aoki 1967, Martin 1967, Sondik 1971, Bertsekas and Shreve 1978, Hee 1978, Monahan 1982).”
Offline policy evaluation across representations with applications to educational games (2014, background): “Although the most general representations such as Predictive State Representations (PSRs) [15] may be capable of capturing the environment, learning all the parameters of a complex PSR or POMDP [22] model directly may not always be the best ap-”
Robotic mapping: a survey (2003, methodology): “In the field of robot mapping, the single dominating scheme for integrating such temporal data is known as Bayes filter [45], which is highly related to Kalman filters [49, 64], hidden Markov models [78], dynamic Bayes networks [82], and partially observable Markov decision processes [48, 59, 66, 93”
Anytime Point-Based Approximations for Large POMDPs (2006, methodology): “A commonly used technique for speeding up POMDP solving involves selecting a ﬁnite set of belief points and performing value backups on this set (Sondik, 1971; Cheng, 1988; Lovejoy, 1991a; Hauskrecht, 2000; Zhang & Zhang, 2001).”
An Online Spectral Learning Algorithm for Partially Observable Nonlinear Dynamical Systems (2011, background): “If the observations are discrete, then Hidden Markov Models (HMMs) (Rabiner 1989) or, in the controlled setting, Partially Observable Markov Decision Processes (POMDPs) (Sondik 1971) can be used to represent belief as a discrete distribution over latent states.”