Learning the structure of Factored Markov Decision Processes in reinforcement learning problems

By Thomas Degris, Olivier Sigaud, Pierre-Henri Wuillemin. 2006

130 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Management Information Systems, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Information Systems

In Reinforcement Learning

Citations in the corpus

Investigating Contingency Awareness Using Atari 2600 Games (2012, background): “For example, our notion of contingency awareness naturally extends to domains in which the observation space is large and factored (Degris, Sigaud, and Wuillemin 2006).”
Dyna-Style Planning with Linear Function Approximation and Prioritized Sweeping (2008, background): “Degris, Sigaud and Wuillemin (2006) developed a version of Dyna based on approximations in the form of dynamic Bayes networks and decision trees.”
Off-policy Model-based Learning under Unknown Factored Dynamics (2015, background): “Degris et al. (2006) proposed a general framework for iteratively learning the dependency structure (this work falls within this framework), yet no theoretical results were presented for their approach.”
A Physics-Based Model Prior for Object-Oriented MDPs (2014, unknown): “The core strength of this approach, in contrast to for example Factored-MDPs (Degris et al., 2006), is that statistical dependency between attributes is no longer stationary but rather on a function which is evaluated at each query state.”
Exploiting structure and utilizing agent-centric rewards to promote coordination in large multiagent systems (2013, methodology): “One popular application of exploiting structure involves factorization within Markov Decision Processes (MDPs) that are extremely large or intractable [7, 37].”
Efficient Structure Learning in Factored-State MDPs (2007, background): “Degris, Sigaud, & Wuillemin (2006) introduce a model-based algorithm that incrementally builds a decisiontree representation of state transitions.”
Generalized model learning for Reinforcement Learning on a humanoid robot (2010, methodology): “[12] use decision trees to learn a model of the MDP.”
Learning and Using Models (2012, methodology): “This approach is taken by many algorithms (Brafman and Tennenholtz, 2001; Hester and Stone, 2009; Degris et al, 2006).”
Exploiting Structure and Agent-Centric Rewards to Promote Coordination in Large Multiagent Systems (2014, methodology): “One popular application of exploiting structure involves factorization within Markov Decision Processes (MDPs) that are extremely large or intractable [7, 37].”
The Bottleneck Simulator: A Model-based Deep Reinforcement Learning Approach (2018, background): “…MDP (for example, by factorizing the state transition model) has also been investigated extensively in the literature (Boutilier et al., 1999; Degris et al., 2006; Strehl et al., 2007; Ross & Pineau, 2008; Bellemare et al., 2013; Wu et al., 2014; Osband & Van Roy, 2014; Hallak et al., 2015).”
Real time targeted exploration in large domains (2010, methodology): “Many methods such as SPITI [6] use ǫ-greedy exploration, where the agent takes what it thinks is the optimal action most of the time, but takes a random action ǫ of the time.”
Safe Policy Improvement with Baseline Bootstrapping in Factored Environments (2019, methodology): “These approaches include methods to search for the underlying structure with and without guarantees of sample complexity (Strehl, Diuk, and Littman 2007; Diuk, Li, and Lefﬂer 2009; Chakraborty and Stone 2011; Degris, Sigaud, and Wuillemin 2006).”