State of the Art—A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms

By G. Monahan. 1982

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Informing sequential clinical decision-making through reinforcement learning: an empirical study (2010, background): “…without precedent in the reinforcement learning literature, where the Partially Observable Markov Decision Process (POMDP) paradigm was developed to handle cases in which the observation does not fully reveal the state of the system (Smallwood and Sondik 1973; Monahan 1982; Kaelbling et al. 1998).”
Task Agnostic Continual Learning via Meta Learning (2019, background): “This information, however, is not available when the ground truth y t depends not only on the observation x t but also on some hidden task (or context) variable T t : y t = f ( x t , T t ) , a common situation in partially observable environments (Monahan, 1982; Cassandra et al., 1994).”
Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning (2016, background): “It has been shown that the belief state is sufﬁcient for optimal control (Monahan, 1982), so that the objective is to ﬁnd π ∗ : b → a that maximizes the expected future return.”
Learning strategies in table tennis using inverse reinforcement learning (2014, background): “Modeling such hidden and uncertain information in the state space leads to the formulation of partial observable MDPs (PoMDPs, (Monahan, 1982)).”
Exploration Bonuses and Dual Control (1996, background): “Performing dynamic programming to solve a POMDP is highly computationally expensive (Monahan, 1982; Littman, 1996).”
Reinforcement Learning: An Introduction (1998, methodology): “17.3 A good presentation of the POMDP approach is given by Monahan (1982).”
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey (2018, background): “can observe a part of the system state, and thus Partially Observable Markov Decision Processes (POMDPs) [15] can”
Long Document Classification From Local Word Glimpses via Recurrent Attention Learning (2019, methodology): “Since ourmodel uses the hard attentionmechanism to smartly extract key words or sentences in a long document, it can be modeled by a typical Partially Observable Markov Decision Process (POMDP) [36] with a reward strategy followed by RAM [16].”
Hidden-Mode Markov Decision Processes for Nonstationary Sequential Decision Making (2001, methodology): “…brie y discuss how the latter can be achieved by a variant ofthe Baum-Welch algorithm, and then develop a similar algorithm for HM-MDP.3.1 Learning a POMDP ModelTraditional research in POMDP (Monahan 1982; Lovejoy 1991) assumes a knownenvironment model and is concerned with nding an optimal policy.”
Reinforcement Learning: A Tutorial Survey and Recent Advances (2009, background): “In such problems, it can be shown (Lovejoy, 1991; Sondik, 1978; Smallwood and Sondik, 1973; Monahan, 1982; White III and Scherer, 1989; Lin et al., 2004) that it is sufficient to keep a probability distribution called the belief state distribution of the current state.”
Equivalence notions and model minimization in Markov decision processes (2003, methodology): “The reduced model can then be solved using a standard POMDP algorithm [Monahan, 1982][Littman, 1994][Cassandraet l., 1997][Zhang and Zhang, 2001].”