Algorithm-Directed Exploration for Model-Based Reinforcement Learning in Factored MDPs

By Carlos Guestrin, Relu Patrascu, Dale Schuurmans. 2002

67 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Reinforcement Learning in Healthcare: A Survey (2019, methodology): “ows exponentially in the representation size. A large number of methods has been proposed to employ factored representation of MDP models for improving learning efﬁciency for either model-based [60], [61] or model-free RL problems [62]. A more challenging issues is how to learn this compact structur”
Exploration in Metric State Spaces (2003, background): “For each of these approaches, there are now at least plausible heuristics, and sometimes formal analysis, for problems of planning [2] and learning.”
Structure Learning for Safe Policy Improvement (2019, methodology): “…the required precision and the expected level of confidence 1− δ. Extensions of this algorithm have been proposed for FMDPs with different assumptions regarding the prior knowledge about the structure of the DBN [Guestrin et al., 2002; Strehl, 2007; Diuk et al., 2009; Chakraborty and Stone, 2011].”
Provable Rich Observation Reinforcement Learning with Combinatorial Latent States (2021, background): “Two MDPs.”
Learning and Using Models (2012, background): “Similar to the extension of E 3 to DBN-E 3 , R-MAX can be extended to FACTORED- R-MAX for factored domains where the structure of the DBN transition model is given (Guestrin et al, 2002).”
Learning the structure of Factored Markov Decision Processes in reinforcement learning problems (2006, methodology): “Indirect rl methods using planning algorithms in fmdp s are described in Guestrin et al. (2002).”
Off-policy Model-based Learning under Unknown Factored Dynamics (2015, background): “Early works assumed the dependency structure is known (Guestrin et al., 2002; Kearns & Koller, 1999).”
Sample Complexity Bounds of Exploration (2012, methodology): “This insight can be used to derive PAC-MDP algorithms for factored-state MDPs (Guestrin et al, 2002; Kearns and Koller, 1999; Strehl, 2007a).”
Safe Policy Improvement with Baseline Bootstrapping in Factored Environments (2019, background): “Factored RL algorithms can have a sample complexity that scales only polynomially in the number parameters of the FMDP (Kearns and Koller 1999; Guestrin, Patrascu, and Schuurmans 2002; Strehl 2007).”