Computing Optimal Policies for Partially Observable Decision Processes Using Compact Representations

By Craig Boutilier, David Poole. 1996

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, neutral): “However, memories of previous events can help to deal with partially observable Markov decision problems (POMDPs) (e.g., Boutilier & Poole, 1996; Jaakkola, Singh, & Jordan, 1995; Kaelbling et al., 1995; Kimura, Miyazaki, & Kobayashi, 1997; Lin, 1993; Littman, Cassandra, & Kaelbling, 1995; McCallum,…”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, background): “This triggered work on partially observable Markov decision problems (POMDPs) [223, 222, 227, 204, 205, 206, 316, 148, 278, 122, 152, 25, 114, 160, 126, 308, 309, 183].”
Finding Approximate POMDP solutions Through Belief Compression (2011, unknown): “In the same vein, state abstraction (Boutilier & Poole, 1996) has been explored to take advantage of factored state spaces, and of particular interest is the algorithm of Hansen and Feng (2000) which can perform state abstraction in the absence of a prior factorization.”
Reinforcement learning for factored Markov decision processes (2002, methodology): “Some ([Boutilier and Poole 1996; Dearden and Boutilier 1997; Boutilier, Dearden, and Goldszmidt 2000]) consider the case when the actions can be explicitly enumerated and give an algorithm for finding the exact value function given a compact representation of an MDP.”
Decision-Theoretic Planning: Structural Assumptions and Computational Leverage (1999, background): “Boutilier and Poole (1996) show how classic history-independentmethods for solving POMDPs, based on conversion to a FOMDP with belief states, can ex-ploit the types of structured representations described here.”
Stochastic dynamic programming with factored representations (2000, methodology): “4See [16] for more detailed investigations of this type.”
Point-Based Value Iteration for Continuous POMDPs (2006, background): “, 2001) and one must resort to the exploitation of model-specific structural properties to obtain approximate scalable algorithms for POMDPs with large state spaces (Boutilier and Poole, 1996; Roy and Gordon, 2003; Poupart and Boutilier, 2003, 2005) and complex policy spaces (Pineau et al.”
Abstraction and Approximate Decision-Theoretic Planning (1997, background): “Recently,Markov decision processes (MDPs) [26, 54, 45] have been proposed as a semantic and computational framework in which to formulate DTP problems [15, 2, 56, 10, 8, 13, 11].”
Solving Relational and First-Order Logical Markov Decision Processes: A Survey (2012, methodology): “IDP techniques are conceptually easy to extend towards POMDPs (Boutilier and Poole, 1996).”
Algorithms for Sequential Decision Making (1996, background): “The C-Buridan [50, 49] and structured policy iteration [25] algorithms solve partially observable problems using compact representations directly.”
Planning and Acting in Partially Observable Stochastic Domains (1998, background): “Algorithms for manipulating compact (or factored) pomdps have begun to appear [12, 6]|this is a promising area for future research.”
HQ-Learning (1997, methodology): “Boutilier and Poole (1996) use Bayesian networks to represent POMDPs, and use these more compact models to accelerate policy computation. Parr and Russell (1995) use gradient descent methods on a continuous representation of the value function.”
Planning, Learning and Coordination in Multiagent Decision Processes (1996, background): “27This is a toy domain in which a robot is supposed to get coffee from a coffee shop across the street, can get wet if it is raining unless it has an umbrella, and is rewarded if it brings coffee when the user requests it, and penalized (to a lesser extent) if it gets wet [9, 10].”
Bayesian Reinforcement Learning in Factored POMDPs (2018, methodology): “The dynamics of a F-POMDP are then defined as follows D(s ′,o |s,a) = [ ∏ s ′i ∈s ′ Pθa (s ′i |PVGa (s))][ ∏ oi ∈o Pθa (oi |PVGa (s ′))], Some approaches are able to exploit the factorization of F-POMDPs, which typically leads to better solution [3].”