A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms

By G. Monahan, George E. MONAHANt. 2007

496 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Informing sequential clinical decision-making through reinforcement learning: an empirical study (2010, background): “This problem is not without precedent in the reinforcement learning literature, where the Partially Observable Markov Decision Process (POMDP) paradigm was developed to handle cases in which the observation does not fully reveal the state of the system (Smallwood and Sondik, 1973; Monahan, 1982; Kae”
Modeling of risk-based inspection, maintenance and life-cycle cost with partially observable Markov decision processes (2005, methodology): “The model he used was called the partially observable Markov decision process (POMDP), which has been studied and used by many researchers (e.g. Eckles 1968, Monahan 1982, Ben-Akiva et al. 1991).”
Perseus: Randomized Point-based Value Iteration for POMDPs (2005, methodology): “Another set of exact POMDP value iteration algorithms do not focus on searching in the belief space, but instead consider enumerating all possible vectors of HVn and then pruning useless vectors (Monahan, 1982; Cassandra, Littman, & Zhang, 1997).”
Online Planning Algorithms for POMDPs (2008, background): “Partially observable Markov decision processes (POMDPs) provide a general framework for acting in partially observable environments (Astrom, 1965; Smallwood & Sondik, 1973; Monahan, 1982; Kaelbling, Littman, & Cassandra, 1998).”
Reinforcement Learning: A Tutorial Survey and Recent Advances (2009, unknown): “In such problems, it can be shown (Lovejoy, 1991; Sondik, 1978; Smallwood and Sondik, 1973; Monahan, 1982; White III and Scherer, 1989; Lin et al., 2004) that it is sufficient to keep a probability distribution called the belief state distribution of the current state.”
Learning strategies in table tennis using inverse reinforcement learning (2014, background): “Modeling such hidden and uncertain information in the state space leads to the formulation of partial observable MDPs (PoMDPs, (Monahan 1982)).”
Anytime Point-Based Approximations for Large POMDPs (2006, unknown): “The Partially Observable Markov Decision Process (POMDP) ( ¨Astrom, 1965; Sondik, 1971; Monahan, 1982; White, 1991; Lovejoy, 1991b; Kaelbling, Littman, & Cassan-dra, 1998; Boutilier, Dean, & Hanks, 1999) has emerged as possibly the most general representation for (single-agent) planning under…”
Multi-Agent Reinforcement Learning: A Review of Challenges and Applications (2021, background): “A partially-observable Markov decision process (POMDP) is a generalization of an MDP that considers the uncertainty regarding the state of a Markov process, allowing a state information acquisition [10].”
Planning algorithms (2006, background): “For POMDPs, a substantial body of work has been developed in operations research and stochastic control theory [567, 658, 717, 900] and more recently in artificial intelligence [497, 650, 651, 740, 775, 794, 806, 808, 838, 1002, 1003].”
An exploration strategy for non-stationary opponents (2017, methodology): “HM-MDPs need an offline learning phase and they are computationally complex since they need to solve a POMDP [38] to obtain a policy to act.”
A Survey on Machine-Learning Techniques in Cognitive Radios (2013, background): “It is an extensi on of the partially observable Markov decision process (POMDP) [98], [99] framework and a specific ase of a partially observable stochastic game (POSG) [100].”
Point-Based Value Iteration for Continuous POMDPs (2006, methodology): “Now, using a reasoning parallel to that of the enumeration phase of the Monahan’s algorithm (Monahan, 1982), we have at most |A|M |O| different αn-functions (fixing the action, we can select one of the M α j a,o-functions for each one of the observations) and, thus, {αn}i is a discrete set.”