Learning Policies for Partially Observable Environments: Scaling Up

By Michael L. Littman, Anthony R. Cassandra, Leslie Pack Kaelbling. 1997

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, background): “However, memories of previous events can help to deal with partially observable Markov decision problems (POMDPs) (e.g., Schmidhuber, 1991c; Ring, 1991, 1993, 1994; Williams, 1992a; Lin, 1993; Teller, 1994; Schmidhuber, 1995; Kaelbling et al., 1995; Littman et al., 1995; Boutilier and Poole, 1996; L”
Reinforcement Learning: A Survey (1996, methodology): “The resulting formal model is called a partially observable Markov decision process or POMDP.”
Optimal and Approximate Q-value Functions for Decentralized POMDPs (2008, methodology): “5.1 QMDP QMDP was originally proposed to approximately solve POMDPs by Littman, Cassandra, and Kaelbling (1995), but has also been applied to Dec-POMDPs (Emery-Montemerlo et al., 2004; Szer et al., 2005).”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, background): “This triggered work on partially observable Markov decision problems (POMDPs) [223, 222, 227, 204, 205, 206, 316, 148, 278, 122, 152, 25, 114, 160, 126, 308, 309, 183].”
Deep Variational Reinforcement Learning for POMDPs (2018, background): “One option would be to copmute a weighted average over K policies and value functions that each take in a single particle value (z t , h k t ) – this however would ignore the uncertainty in the latent state after the next action (Littman et al., 1995).”
Sidekick agents for sequential planning problems (2013, background): “[40], weights the Q*MDP values for the states and actions of the MDP by the state probabilities given by b.”
Anytime Point-Based Approximations for Large POMDPs (2006, background): “Figure 15 looks at the policy obtained when solving this same problem using the QMDP heuristic.”
A video game description language for model-based or interactive learning (2013, background): “• 89-state maze [16], with stochastic actions and observations; see Figure 5.”
A Survey on Machine-Learning Techniques in Cognitive Radios (2013, methodology): “In particular, several RL algorithms were shown to provide efficient nearoptimal solutions to the POMDP's, yet with low complexity [54], [102], [103].”
A Concise Introduction to Decentralized POMDPs (2016, methodology): “As a result of its similarity to an approach to approximate POMDPs [Littman et al., 1995], this approach is called the QMDP heuristic. Intuitively, it should lead to an admissible heuristic because the MDP value function assumes that the state is observable. In a similar fashion, it is also possible”
Monte Carlo POMDPs (1999, methodology): “We usedt to denote the history leading up to timet: dt := fot; at 1; ot 1; at 2; : : : ; a0; o0g (1) The fundamental problem in POMDPs is to devise a policy for action selection that maximizes reward.”
The Actor Search Tree Critic (ASTC) for Off-Policy POMDP Learning in Medical Decision Making (2018, background): “Most exhaustive algorithms for POMDPs are dedicated to learning either lower bound [12, 16] or upper bound [17, 18, 19] of v∗(b) by maintaining a subset of the aforementioned hyperplanes.”
Reinforcement Learning with Limited Reinforcement : Using Bayes Risk for Active Learning in POMDPs Finale (2012, methodology): “As expressed in equation 9, the Bayes risk criterion is simil ar to theQMDP heuristic [29], which uses the approximation V (b) = max ∑”
Recurrent policy gradients (2010, background): “In [29] the human performance of 29 steps and 100% is highlighted, which again underlines the difficulty of the task.”
Resource Management with Deep Reinforcement Learning (2016, background): “The RL paradigm can in principle deal with such situations of partial observability by casting the decision problem as a POMDP [9].”
Policy-Gradient Algorithms for Partially Observable Markov Decision Processes (2003, methodology): “A simple approach is Linear-Q which defines Q(b, u) = φu ·b [Littman et al., 1995].”