Generalizing plans to new environments in relational MDPs

By Carlos Guestrin, Daphne Koller, Chris Gearhart, Neal Kanodia. 2003

220 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Fitted Q-Learning for Relational Domains (2020, background): “A notable exception is by Guestrin et al. (2003), who developed approximate learning for RMDPs by representing and learning value functions in a stage-wise manner.”
Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation (2016, background): “[11], who describe an object-based representation in the context of planning.”
Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “One popular formulation, called Hidden Parameter MDPs (HiP-MDPs) (Doshi-Velez and Konidaris, 2013), models the environment as a single fixed unknown task that may, for example, be drawn from a stationary distribution over tasks at the beginning of each episode.”
Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics (2017, positive): “However, neither OO-MDPs nor R-MDPs deﬁne a transition function with an explicit OR of possible causes, and traditionally transition functions have not been learned in these models.”
Neural Predictive Belief Representations (2018, background): “Indeed, representations that encode important parts of the state have led to improved performance in RL tasks, both with model-based and value-based methods (Guestrin et al., 2003; Diuk et al., 2008; Boots et al., 2011; Levine et al., 2016; Higgins et al., 2017; Karkus et al., 2018).”
Approximate Policy Iteration with a Policy Language Bias: Solving Relational Markov Decision Processes (2011, methodology): “This has been considered by Guestrin et al. (2003a), where a class-based MDP and value function representation was used to compute an approximate value function that could generalize across diﬀerent sets of objects.”
Strategic Object Oriented Reinforcement Learning (2018, background): “However, extending these methods to large MDPs remain difficult.”
Neural Logic Reinforcement Learning (2019, background): “…cases with environment models known, variations of traditional MDP solvers such as dynamic programming (Boutilier et al., 2001), linear programming (Guestrin et al., 2003) and heuristic greedy searching (Yoon et al., 2002) were employed to optimise policies in training tasks that can be…”
RTS Games and Real-Time AI Research (2003, methodology): “Nevertheless, it has recently been used as test–vehicle for MDP related planning research [10].”
Transfer in variable-reward hierarchical reinforcement learning (2008, unknown): “Guestrin et al. (2003) use linear programming techniques to solve relational MDPs to compute a set of value functions from one MDP that can be directly used in another.”
Toybox: A Suite of Environments for Experimental Evaluation of Deep Reinforcement Learning (2019, methodology): “• State Representation Operating over pixel representations like Atari games means the agent experiences a huge state space, but the underlying objects deﬁning those sensory representations can often be represented more compactly (Guestrin et al., 2003; Diuk et al., 2008; Kansky et al., 2017).”
Practical solution techniques for first-order MDPs (2009, unknown): “For a few years immediately succeeding the publication of the symbolic dynamic programming solution (Boutilier et al., 2001) to relational ly specified MDPs, this domain-independent non-grounding approach was disparaged as being unrealistic for practical applications due to the complexity of value f”
First Order Decision Diagrams for Relational MDPs (2007, background): “, 2004], approximate linear programming for FOMDPs [Guestrin et al., 2003; Sanner and Boutilier, 2005], approximate policy iteration [Fern et al.”
Learning to Win: Case-Based Plan Selection in a Real-Time Strategy Game (2005, methodology): “Several researchers have addressed classic board games, beginning with Arthur Samuel’s (1959) rote learning approach for playing checkers.”
Value Functions for RL-Based Behavior Transfer: A Comparative Study (2005, background): “Another approach (Guestrin et al. 2003) uses linear programming to determine value functions for classes of similar agents.”
Concurrent Hierarchical Reinforcement Learning (2005, methodology): “Section 4 describes the methods used to handle the very large SMDPs that we face: we use linear function approximators combined with relational feature templates [Guestrin et al., 2003], and for action selection, we use coordination graphs [Guestrin et al., 2002].”