Model-based function approximation in reinforcement learning

By Nicholas K. Jong, Peter Stone. 2007

74 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Sergey Levine
2. Pieter Abbeel
3. Peter Stone
4. Nan Jiang
5. Lihong Li
6. Jan Peters
7. Shimon Whiteson
8. Ofir Nachum

In Computational Theory and Mathematics

1. Lihong Li
2. Nan Jiang
3. Jan Peters
4. Sergey Levine
5. Mengdi Wang
6. Shie Mannor
7. Oliver Kroemer
8. Marcello Restelli

Citations in the corpus

• Doubly Robust Off-policy Value Evaluation for Reinforcement Learning (2015, methodology): “To generalize, one can either apply function approximation to fitting (cid:99) M (Jong & Stone, 2007; Gr¨unew¨alder et al., 2012), or to fitting the value function directly (Bertsekas & Tsitsiklis, 1996; Sutton & Barto, 1998; Dann et al., 2014).”
• Minimax-Optimal Off-Policy Evaluation with Linear Function Approximation (2020, methodology): “…to tackle MDPs with infinite or continuous state spaces, one often needs various forms of function approximation, and many methods like fitted Q-iteration and least square policy iteration were developed (Jong & Stone, 2007; Lagoudakis & Parr, 2003; Grunewalder et al., 2012; Fonteneau et al., 2013).”
• Sample Complexity Bounds of Exploration (2012, background): “A number of novel algorithms are motivated and work well in practice (Jong and Stone, 2007; Li et al, 2009; Nouri and Littman, 2009, 2010), all of which incorporate various notion of knownness in guiding exploration.”
• A Non-Parametric Approach to Dynamic Programming (2011, methodology): “The DSDP algorithm has also been the basis for continuous-state policy evaluation algorithms [26, 27].”
• Off-policy Model-based Learning under Unknown Factored Dynamics (2015, background): “Interestingly, the Flat (model-based) approach appears to be more sample ef-ficient than MFMC , which is in line with observations that model-based RL is more efficient than model-free RL (Hester & Stone, 2009; Jong & Stone, 2007).”
• Transfer of samples in batch reinforcement learning (2008, methodology): “In particular, we follow the kernel-based approximation proposed in ( Jong & Stone, 2007 ).”
• Learning and Using Models (2012, methodology): “We also evaluated FITTEDR-MAX (Jong and Stone, 2007), which is an extension of R-MAX for continuous domains, and CONTINUOUS TEXPLORE, an extension of TEXPLORE to continuous domains that uses regression trees to model the continuous state instead of discrete”
• Planning and Execution using Inaccurate Models with Provable Guarantees (2020, methodology): “We chose two function approximators for the learned residual dynamics to account for model learning approaches that use global function approximators such as neural networks (NN) [14], and local function approximators such as K-nearest neighbor regression (KNN) [25, 16].”
• Doubly Robust Off-policy Evaluation for Reinforcement Learning (2015, unknown): “To generalize over the state (and action) space, one can either choose to apply function approximators to fitting M (Jong & Stone, 2007; Grünewälder et al., 2012), or to fitting the value function directly 2 (Bertsekas & Tsitsiklis, 1996; Sutton & Barto, 1998; Dann et al., 2014).”