Stochastic dynamic programming with factored representations

By Craig Boutilier, Richard Dearden, Moisés Goldszmidt. 2000

497 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “These cases can be formalized through the Factored MDP assumption (Kearns and Koller, 1999; Boutilier, Dearden, and Goldszmidt, 2000): Definition 7 (Factored MDP Assumption): Each state is composed of n factors s := s1, s2, ..., sn ∈ S ⊆ S1 × S2 × ... × Sn and each action can be composed of m…”
Dyna-Style Planning with Linear Function Approximation and Prioritized Sweeping (2008, methodology): “We introduce two versions of prioritized sweeping with linear Dyna and briefly illustrate their performance empirically on the Mountain Car and Boyan Chain problems.”
Efficient Solution Algorithms for Factored MDPs (2003, methodology): “To better understand this difference, we evaluated both our rule-based approximate linear programming algorithm and Apricodd in two problems, Linear and Expon, designed by Boutilier et al. (2000) to illustrate respectively the best-case and the worst-case behavior of their algorithm.”
Near-optimal Reinforcement Learning in Factored MDPs (2014, background): “Factored MDPs (FMDPs) [7], whose transitions can be represented by a dynamic Bayesian network (DBN) [8], are one e ective way to represent these structured MDPs compactly.”
A decision network account of reasoning about other people's choices. (2015, methodology): “Namely, decision nets use a feature-based representation and MDPs use a a state-based representation (Boutilier et al., 1999; Boutilier et al., 2000; Poole & Mackworth, 2010).”
Learning and Using Models (2012, methodology): “It plans on this model using Structured Value Iteration (Boutilier et al, 2000) and uses ε-greedy exploration.”
Bisimulation Metrics for Continuous Markov Decision Processes (2011, background): “In practice, however, MDPs are rarely represented explicitly; instead, researchers usually work with factored representations [9], wherein the state space is represented by a family of state variables.”
Decision-Theoretic Planning: Structural Assumptions and Computational Leverage (1999, background): “What might beviewed as best- and worst-case behavior is also described in (Boutilier et al., 1999).”
Symbolic Dynamic Programming for First-Order MDPs (2001, methodology): “We then formulate situation calculus domain axioms using these deterministic choices [Bacchus et al., 1995; Reiter, 2001; Boutilier et al., 2000b].”
Minimal Sufficient Explanations for Factored Markov Decision Processes (2009, background): “We focus on factored MDPs (Boutilier, Dearden, and Goldszmidt 2000) in which a set of state variables define the state space, with the state of the MDP determined by the values of all state variables.”
Equivalence notions and model minimization in Markov decision processes (2003, methodology): “[Boutilier et al., 2000] describe variants of policy iteration, successive approximation, and value iteration designed to work on factored MDP representations, called structured policy iteration(SPI), structured successive approximation(SSA), andstructured value iteration(SVI), respectively.”
Solving Relational and First-Order Logical Markov Decision Processes: A Survey (2012, unknown): “Boutilier et al (2000) introduced structured value and policy iteration algorithms (SVI and SPI) based on propositional trees for value functions and policies and dynamic Bayesian networks for representing the transition function.”