Abstraction Selection in Model-based Reinforcement Learning

By Nan Jiang, Alex Kulesza, Satinder Singh. 2015

86 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

DeepMDP: Learning Continuous Latent Space Models for Representation Learning (2019, methodology): “Furthermore, state aggregation techniques, whether based on bisimulation or other methods (Abel et al., 2017; Li et al., 2006; Singh et al., 1995; Givan et al., 2003; Jiang et al., 2015; Ruan et al., 2015), suffer from poor compatibility with function approximation methods.”
Doubly Robust Off-policy Value Evaluation for Reinforcement Learning (2015, background): “Such a bias is in general hard to quantify from data, thus breaks the credibility of estimations given by regression based approaches (Farahmand & Szepesvári, 2011; Marivate, 2015; Jiang et al., 2015).”
Information-Theoretic Considerations in Batch Reinforcement Learning (2019, background): “It is known that certainty-equivalence with Q ⋆ -irrelevant abstraction is consistent and enjoys polynomial sample complexity when each state-action pair receives enough data (Li, 2009; Hutter, 2014; Jiang et al., 2015; Abel et al., 2016; Jiang, 2018).”
Offline Policy Selection under Uncertainty (2020, methodology): “Previously, this term has been used to refer to state abstraction selection (Jiang, 2017; Jiang et al., 2015) as well as learning algorithm and feature selection (Foster et al.”
Small-sample reinforcement learning: Improving policies using synthetic data (2017, unknown): “There is work on how to select the best abstraction of the state space when limited data is available [8] and methods for quantifying how uncertainty from limited data propagates through the learning al-gorithm [6], but there is a need for further research in extracting good policies from small…”
Block Contextual MDPs for Continual Learning (2021, unknown): “From Lemma 3 in Jiang et al. (2015).”
Value Preserving State-Action Abstractions (2020, methodology): “Recent work has continued to clarify the conditions under which state abstractions preserve value in MDPs (Jiang et al., 2015a; Abel et al., 2016), non-Markovian environments (Hutter, 2014), planning algorithms (Hostetler et al., 2014; Jiang et al., 2014; Anand et al., 2015) and lifelong RL (Abel et”
Near Optimal Behavior via Approximate State Abstraction (2016, background): “Jiang [19] analyze the problem of choosing between two candidate abstractions.”
A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms (2019, background): “Each skill can use its own internal skill-specific abstraction [90, 201, 400, 69, 178] that allows it to focus on only relevant objects and state features, decomposing a problem that”
FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPs (2020, background): “Note that Ortner et al. (2014); Jiang et al. (2015) consider a form of representation learning, abstraction selection, but the former scales linearly with the number of candidate abstractions, while the latter does not address exploration.”
Provably efficient RL with Rich Observations via Latent State Decoding (2019, background): “The very few results that come with finite sample guarantees can only handle a small number of candidate abstractions (Hallak et al., 2013; Ortner et al., 2014; Jiang et al., 2015).”
Regret Minimization Experience Replay in Off-Policy Reinforcement Learning (2021, background): “With the Lemma 3 in [30], Because π ( a s ) = exp( Q ( s,a )) a 0 exp( Q ( s,a 0 )) , π is continuous with respect to Q , then we have The continuity of π and transition function T guarantees Therefore, Notably, 2 ( s ) π k ( a s ) ≤ d π k ( s, a ) , the existence of d πk ( s,a ) µ ( s,a ) implies…”
An Introduction to Deep Reinforcement Learning (2018, unknown): “The idea of selecting the right features for the task at hand is key in the whole field of machine learning and also highly prevalent in reinforcement learning (see e.g., Munos and Moore, 2002; Ravindran and Barto, 2004; Leffler et al., 2007; Kroon and Whiteson, 2009; Dinculescu and Precup, 2010; Li”
PAC Reinforcement Learning with Rich Observations (2016, background): “Furthermore, most sample complexity results cannot search over large abstraction sets (see e.g. Jiang et al. [9]), limiting their scope.”
The Bottleneck Simulator: A Model-based Deep Reinforcement Learning Approach (2018, methodology): “Numerous algorithms exists for models based on state abstraction (state aggregation) (Bean et al., 1987; Bertsekas & Castanon, 1989; Dean et al., 1997; Dietterich, 2000; Jong & Stone, 2005; Li et al., 2006; Jiang et al., 2015).”