Bounding Performance Loss in Approximate MDP Homomorphisms

By Jonathan Taylor, Doina Precup, Prakash Panangaden. 2008

87 citations recorded by the source corpus.

Subfields: Reinforcement Learning, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Statistical Learning Theory

Citations in the corpus

Value Iteration Networks (2016, background): “, 1999), hierarchical macro-actions (Mann & Mannor, 2014), and state-aggregation (Taylor et al., 2009; Bertsekas, 2012) for reducing the number of iterations required for VI to converge, with the goal of reducing computation time.”
Using Bisimulation for Policy Transfer in MDPs (2010, background): “In particular, we focus on bisimulation metrics (Ferns et al., 2004; Taylor et al., 2009), which measure the long-term behavioral similarity of different states.”
Value Preserving State-Action Abstractions (2020, background): “Recent work has continued to clarify the conditions under which state abstractions preserve value in MDPs (Jiang et al., 2015a; Abel et al., 2016), non-Markovian environments (Hutter, 2014), planning algorithms (Hostetler et al., 2014; Jiang et al., 2014; Anand et al., 2015) and lifelong RL (Abel et”
A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress (2018, methodology): “The method uses an MDP metric quantifying the equivalence between states based on the closeness of the output action-distributions of (stochastic) policy πE at the states [35, 36].”
Autonomous Agents and Multi-agent Systems Manuscript No. Learning Potential Functions and Their Representations for Multi-task Reinforcement Learning (unknown): “Although similar methods have also been applied to transfer learning, most of these learn task-relevant representations for SL [2,5] or RL [18,30,37,40,60,70,75,83]; the latter aim to reduce the state space of the target task, or find good representations for value functions or policies.”
Scalable methods for computing state similarity in deterministic Markov Decision Processes (2019, background): “Taylor, Precup, and Panagaden (2009) overcame this problem by the introduction of lax-bisimulation metrics (deﬁnition and theoretical results provided in the supplemental).”
Automatic Construction of Temporally Extended Actions for MDPs Using Bisimulation Metrics (2011, result): “It is important to note that, unlike in other research, such as transfer learning (see [Taylor and Stone, 2009] for a survey), we do not attempt to provide an entire solution to the new system.”
Offline Reinforcement Learning with Pseudometric Learning (2021, background): “Again as the partitioning induced by a homomorphism is too restrictive to be useful in practice, a number of work has looked into relaxed versions (Ravindran & Barto, 2004; Wolfe & Barto, 2006; van der Pol et al., 2020; Taylor et al., 2008).”