Planning and Acting in Partially Observable Stochastic Domains

By Leslie Pack Kaelbling, Michael L. Littman, Anthony R. Cassandra. 1998

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Memory-based control with recurrent neural networks (2015, background): “This approach has two major disadvantages: The first is the need for a model, and the second is the computational cost that is typically associated with the update of the belief state [8, 23].”
Concrete Problems in AI Safety (2016, background): “at there always exists a reward function in terms of actions and observations that is equivalent to optimizing the true objective function (this involves reducing the POMPD to a belief state MDP, see [76]), often this reward function involves complicated long-term dependencies and is prohibitively h”
ReasoNet: Learning to Stop Reading in Machine Comprehension (2016, background): “Multi-turn reasoning: For complex passages and complex queries, human readers often revisit the given document in order to perform deeper inference after reading a document.”
Deep learning in neural networks: An overview (2014, neutral): “…events can help to deal with partially observable Markov decision problems (POMDPs) (e.g., Boutilier & Poole, 1996; Jaakkola, Singh, & Jordan, 1995; Kaelbling et al., 1995; Kimura, Miyazaki, & Kobayashi, 1997; Lin, 1993; Littman, Cassandra, & Kaelbling, 1995; McCallum, 1996; Otsuka, Yoshimoto, &…”
Learning World Graphs to Accelerate Hierarchical Reinforcement Learning (2019, neutral): “probabilistic planning (Kaelbling et al., 1998), latent embedding of (belief) states (Guo et al.”
Shaping Belief States with Generative Environment Models for RL (2019, background): “A formal way of describing agents in partially observed environments is through the notion of partially observable Markov decision processes [4, 5] (POMDPs).”
Combining Kernel and Model Based Learning for HIV Therapy Selection (2017, methodology): “This decision-making task may be formulated as a Partially Observable Markov Decision Process (POMDP) [12].”
Informing sequential clinical decision-making through reinforcement learning: an empirical study (2010, background): “This problem is not without precedent in the reinforcement learning literature, where the Partially Observable Markov Decision Process (POMDP) paradigm was developed to handle cases in which the observation does not fully reveal the state of the system (Smallwood and Sondik 1973; Monahan 1982; Kaelb”
Acme: A Research Framework for Distributed Reinforcement Learning (2020, background): “From a mathematical perspective, the interaction between an agent and its environment can generally be characterized as either a Markov Decision Process (MDP) (Puterman, 1994) or a Partially Observable MDP (POMDP) (Kaelbling et al., 1998).”
Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels (2020, methodology): “Reinforcement Learning from Images We formulate image-based control as a ﬁnite-horizon partially observable Markov decision process (POMDP) [6, 27], instantiated by an episodic environment with discrete time step t ∈ [1 , T ] , where T is ﬁxed.”
Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability (2017, background): “Such domains are formalized as Partially Observable Markov Decision Processes (POMDPs), defined as 〈S,A, T ,R,Ω,O, γ〉 (Kaelbling et al., 1998).”
Decentralized Likelihood Quantile Networks for Improving Performance in Deep Multi-Agent Reinforcement Learning (2018, background): “Inspired by real-world tasks, partial observable problems are often formalized as Partially Observable MDPs (POMDPs) (Kaelbling et al., 1998).”
Optimal and Approximate Q-value Functions for Decentralized POMDPs (2008, methodology): “This test problem has been frequently used (Nair et al., 2003b; Emery-Montemerlo et al., 2004; Emery-Montemerlo, Gordon, Schneider, & Thrun, 2005; Szer et al., 2005) and is a modification of the (single-agent) tiger problem (Kaelbling et al., 1998).”
Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning (2020, background): “Similarly, it can represent unobserved variables in partially observable Markov decision processes (POMDPs) [32], hidden contexts in hidden-mode Markov decision processes [12], or tasks’ variations in robotics [75, 27, 82].”
Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables (2019, background): “Partially observed MDPs. Adaptation at test time in meta-RL can be viewed as a special case of RL in a POMDP (Kaelbling et al., 1998) by including the task as the un-observed part of the state.”
Learning Where to Attend with Deep Architectures for Image Tracking (2011, background): “In doing so, the new reward becomes the expectation of the immediate reward with respect to the belief state (Kaelbling, Littman, & Cassandra, 1998 ).”