Acting Optimally in Partially Observable Stochastic Domains

By Anthony R. Cassandra, Leslie Pack Kaelbling, Michael L. Littman. 1994

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Acme: A Research Framework for Distributed Reinforcement Learning (2020, background): “The interaction between an agent and its environment can typically be formalized as a (partially-observable) Markov decision process ((PO)MDP Cassandra et al., 1994; Puterman, 1994).”
Task Agnostic Continual Learning via Meta Learning (2019, background): “This information, however, is not available when the ground truth y t depends not only on the observation x t but also on some hidden task (or context) variable T t : y t = f ( x t , T t ) , a common situation in partially observable environments (Monahan, 1982; Cassandra et al., 1994).”
Learning Causal State Representations of Partially Observable Environments (2019, background): “…starts from a generative model of the latent transitions and emission dynamics, using observations to infer beliefs over the unobserved states [1, 5]; and (ii) predictive state representations (PSRs) , where one starts from the history of the process and constructs states through modeling of the…”
A data-driven statistical framework for post-grasp manipulation (2014, background): “POMDPs [5] (Partially Observable Markov Decision Processes) are a general framework to optimally plan under uncertainty by tracking the distribution of a system under some provided state representation.”
Stochastic dynamic programming with factored representations (2000, background): “Partially observable MDPs (POMDPs) [21, 53, 73] are much more computationally demanding than fully observable MDPs.”
On Q-learning Convergence for Non-Markov Decision Processes (2018, methodology): “…the non-Markovian nature of a POMDP is to estimate a Markovian model with a state estimation method [Whitehead and Lin, 1995; Lin and Mitchell, 1992; Cassandra et al. , 1994; Cassandra, 1994] or use a ﬁnite subset of the recent history as a state [McCallum, 1995] to form a k -order MDP ( k -MDP).”
Learning Finite-State Controllers for Partially Observable Environments (1999, background): “The partially observable Markov decision process ( POMDP) provides a formal framework for studying these problems [1, 27, 28, 7, 11, 6, 16].”
Flexible Decomposition Algorithms for Weakly Coupled Markov Decision Problems (1998, methodology): “The value space search algorithm has some similarities to and was inspired by algorithms for partially observable MDPs (POMDPs) (see Lovejoy (1991) for a survey), and in particular, the Witness algorithm ( Cassandra, Kaelbling, & Littman, 1994 ).”
Finding Approximate POMDP solutions Through Belief Compression (2011, unknown): “2.0.2 Partially Observable Markov Decision Processes Partially Observable Markov Decision Processes (POMDPs) are a framework for planning in partially observable worlds (Sondik, 1971; Cassandra et al., 1994 ).”
Offline Meta Reinforcement Learning - Identifiability Challenges and Effective Data Collection Strategies (2021, methodology): “One way to approach the BRL problem is to model R , P as unobserved state variables in a partially observed MDP [POMDP, 3], reducing the problem to solving a particular POMDP instance where the unobserved variables do not change in time.”