Abstraction and Approximate Decision-Theoretic Planning

By Richard Dearden, Craig Boutilier. 1997

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Representation Matters: Offline Pretraining for Sequential Decision Making (2021, background): “Abstraction and Bisimulation Traditionally, representation learning has been framed as learning or identifying abstractions of the state or action space of an environment (Andre and Russell, 2002; Mannor et al., 2004; Dearden and Boutilier, 1997; Abel et al., 2018).”
Efficient Solution Algorithms for Factored MDPs (2003, background): “Boutilier et al. (Boutilier et al., 1995; Dearden & Boutilier, 1997; Boutilier, Dean, & Hanks, 1999; Boutilier et al., 2000) have developed a set of algorithms which can exploit CSI in the transition and reward models to perform efficient (approximate) planning.”
Hierarchical Solution of Markov Decision Processes using Macro-actions (1998, methodology): “These include function approximation [2], reachability analyses [5] and aggregation techniques [7, 3, 4].”
An MDP Approach for Explanation Generation (2007, background): “To evaluate the qualitative change, we instantiate the variables at time t in the DBN, and via probability propagation, we estimate the values at time t+1.”
Near Optimal Behavior via Approximate State Abstraction (2016, methodology): “ state abstraction to Monte Carlo Tree Search and expectimax search, giving value bounds of applying the optimal abstract action in the ground tree(s), similarly to our setting. Dearden and Boutilier [10] also formalize stateabstraction for planning, focusing on abstractions that are quickly compute”
Decision-Theoretic Planning: Structural Assumptions and Computational Leverage (1999, methodology): “This tech-nique is also investigated in (Bonet et al., 1997; Dearden & Boutilier, 1994, 1997; Koenig& Simmons, 1995), and is closely tied to Korf's (1990) LRTA* algorithm.”
Reinforcement learning for factored Markov decision processes (2002, background): “Some ([Boutilier and Poole 1996; Dearden and Boutilier 1997; Boutilier, Dearden, and Goldszmidt 2000]) consider the case when the actions can be explicitly enumerated and give an algorithm for finding the exact value function given a compact representation of an MDP.”
Value Preserving State-Action Abstractions (2020, background): “With a smaller state space, algorithms can learn with less computation, space, and samples (Singh et al., 1995; Dearden and Boutilier, 1997; Dietterich, 2000b,a; Andre and Russell, 2002; Jong and Stone, 2005).”
Equivalence notions and model minimization in Markov decision processes (2003, background): “…algorithms [Baum and Nicholson, 1998][Boutilier and Dearden, 1994][Lin and Dean, 1995] [Boutilieret al., 1995b] [Boutilier and Poole, 1996][Dearden and Boutilier, 1997][Dean and Givan, 1997][Deanet al., 1998]— reasoning directly about the factored representation to find blocks of states…”
Inventing Relational State and Action Abstractions for Effective and Efficient Bilevel Planning (2022, background): “Abstractions offer a mechanism to overcome this intractability, allowing the agent to reason at a higher level about the most salient aspects of a task [2, 3, 4, 5, 6].”
Practical solution techniques for first-order MDPs (2009, methodology): “Then we could use such a value function as a heuristic seed for online search methods such as RTDP (Barto et al., 1993; Dearden and Boutilier, 1997).”
Accelerating Reinforcement Learning through Implicit Imitation (2003, methodology): “We refer the reader to Bertsekas (1987); Boutilier, Dean and Hanks (1999); and Puterman (1994) for further material on MDPs.”