State Abstractions for Lifelong Reinforcement Learning

By David Abel, Dilip Arumugam, Lucas Lehnert, Michael L. Littman. 2018

140 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “A PAC state abstraction is defined such that it achieves correct clustering with high probability with respect to a distribution over learning problems (Abel et al., 2018).”
Representation Matters: Offline Pretraining for Sequential Decision Making (2021, background): “Abstraction and Bisimulation Traditionally, representation learning has been framed as learning or identifying abstractions of the state or action space of an environment (Andre and Russell, 2002; Mannor et al., 2004; Dearden and Boutilier, 1997; Abel et al., 2018).”
Continual Representation Learning for Biometric Identification (2020, background): “Existing continual learning works focus on general object recognition [7, 29, 48], object detection [16, 54]), image generation [33, 68], reinforcement learning tasks [1, 27, 72] and unsupervised learning tasks [12].”
Modular Lifelong Reinforcement Learning via Neural Composition (2022, background): “Other forms of hierarchical RL have learned state abstractions (Dayan & Hinton, 1993; Dietterich, 2000; Vezhnevets et al., 2017; Abel et al., 2018).”
Non-Stationary Markov Decision Processes a Worst-Case Approach using Model-Based Reinforcement Learning (2019, background): “In the literature, this problem is often linked to the setting of Lifelong Reinforcement Learning (LRL) [Silver et al., 2013, Abel et al., 2018a,b] and learning in non-stationary environments [Choi et al., 1999, Jaulmes et al., 2005, Hadoux, 2015].”
Value Preserving State-Action Abstractions (2020, background): “…continued to clarify the conditions under which state abstractions preserve value in MDPs (Jiang et al., 2015a; Abel et al., 2016), non-Markovian environments (Hutter, 2014), planning algorithms (Hostetler et al., 2014; Jiang et al., 2014; Anand et al., 2015) and lifelong RL (Abel et al., 2018).”
Learning Markov State Abstractions for Deep Reinforcement Learning (2021, background): “3 Even worse, if the agent’s policy changes—such as during learning—this discrepancy can cause RL algorithms with bounded sample complexity in the ground MDP to make an arbitrarily large number of mistakes in the abstract MDP (Abel et al., 2018).”
A Memorizing and Generalizing Framework for Lifelong Person Re-Identification (2023, background): “Thanks to the impressive progresses in deep neural networks, lifelong learning has regained the spotlight and is widely employed in various vision and learning tasks, such as object recognition [15], [43], object detection [44], image generation [45], reinforcement learning [46], [47], unsupervised ”
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, background): “Other work studied state abstractions from a theoretical perspective in the lifelong setting (Abel et al., 2018).”