CORL: A Continuous-state Offset-dynamics Reinforcement Learner

By Emma Brunskill, Bethany R. Leffler, Lihong Li, Michael L. Littman, Nicholas A. Roy. 2008

25 citations recorded by the source corpus.

Continuous state spaces and stochastic, switching dynamics characterize a number of rich, real-world domains, such as robot navigation across varying terrain. We describe a reinforcement-learning algorithm for learning in these domains and prove for certain environments the algorithm is probably approximately correct with a sample complexity that scales polynomially with the state-space dimension. Unfortunately, no optimal planning techniques exist in general for such problems; instead we use fitted value iteration to solve the learned MDP, and include the error due to approximate planning in our bounds. Finally, we report an experiment using a robotic car driving over varying terrain to demonstrate that these dynamics representations adequately capture real-world dynamics and that our algorithm can be used to efficiently solve such problems.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Never-Ending Learning (2015, background): “Work on active and proactive learning (Tong and Koller 2001; Donmez and Carbonell 2008) and on exploitation/exploration tradeoffs (Brunskill et al. 2012) presents strategies by which learning agents can collect optimal training data from their environment.”
Knows what it knows: a framework for self-aware learning (2008, methodology): “Action Outcomes with Gaussian Distributions – Learning the parameters of a Gaussian can be done using a variant of the coin-learning algorithm, and multiple actions can be dealt with using input-partition, just like in the flat MDP case (Brunskill et al., 2008).”
Reinforcement Learning in Finite MDPs: PAC Analysis (2009, methodology): “The analysis techniques are general and have proven useful in analyzing other related algorithms (Asmuth et al., 2008; Brunskill et al., 2008; Leffler et al., 2007; Strehl et al., 2007; Strehl and Littman, 2008a).”
Efficient Exploration in Reinforcement Learning (2017, methodology): “This includes Factored MDPs [4], Metric MDPs [2], Continuous MDPs[1], and MDPs with a Bayesian prior [6].”
Near-Bayesian exploration in polynomial time (2009, background): “, 2003), factored MDPs (Kearns & Koller, 1999) to continuous linear (in state features) domains (Strehl & Littman, 2008b), to a certain class of switching linear systems (Brunskill et al., 2008), and others.”