Gaussian Processes and Reinforcement Learning for Identification and Control of an Autonomous Blimp

By Jonathan Ko, Daniel J. Klein, Dieter Fox, Dirk Hähnel. 2007

225 citations recorded by the source corpus.

Subfields: Aerospace Engineering, Artificial Intelligence, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Artificial Intelligence

In Statistical and Nonlinear Physics

Citations in the corpus

Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020, methodology): “Approaches have included using Gaussian process models for controlling blimps (Ko et al., 2007), and using linear regression (Koppejan and Whiteson, 2009) and locally-weighted Bayesian regression (Bagnell and Schneider, 2001) for helicopter control.”
Reinforcement learning in robotics: A survey (2013, neutral): “, 2005) and an autonomous blimp (Ko et al., 2007).”
Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models (2018, methodology): “…such as Gaussian processes (GPs), are often the model of choice in MBRL, especially in low-dimensional problems where data efficiency is critical [Kocijan et al., 2004, Ko et al., 2007, Nguyen-Tuong et al., 2008, Grancharova et al., 2008, Deisenroth et al., 2014, Kamthe and Deisenroth, 2018].”
Model-Ensemble Trust-Region Policy Optimization (2018, methodology): “An alternative is to use nonparametric models such as Gaussian Processes (GPs) (Rasmussen et al., 2003; Ko et al., 2007; Deisenroth & Rasmussen, 2011).”
Learning and Policy Search in Stochastic Dynamical Systems with Bayesian Neural Networks (2016, neutral): “When the state space is continuous, popular modes for transition functions include Gaussian processes [16, 12, 3], fixed bases such as Laguerre functions [21], and adaptive basis/neural networks [5].”
Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning (2011, methodology): “Typically, the training data for the GP dynamics models are obtained either by motor babbling [9] or by demonstrations [14].”
Dynamics-Aware Unsupervised Discovery of Skills (2019, methodology): “…has been traditionally approached with methods that are well-suited for low-data regimes such as Gaussian Processes (Rasmussen, 2003) showing signiﬁcant data-efﬁciency gains over model-free approaches (Deisenroth et al., 2013; Kamthe & Deisenroth, 2017; Kocijan et al., 2004; Ko et al., 2007).”
Gaussian Processes for Data-Efficient Learning in Robotics and Control (2015, positive): “Instead closed-form Bayesian averaging over inﬁnitely many plausible dynamics models is possible by using non-parametric GPs. Nonparametric GP dynamics models in RL were previously proposed in [17], [30], [46], where the GP training data were obtained from “motor babbling”.”
Analytic moment-based Gaussian process filtering (2009, unknown): “Like Ko et al. (2007) and Ko and Fox (2008), we assume that the transition function and the measurement function can be learned by having access to ground truth observations of the hidden states.”
Gaussian process dynamic programming (2009, methodology): “For example, Ko et al. (2007a) evaluate a method of combining idealized ODEs describing the system dynamics with GP models for the observations originating from the real system.”
A Practical and Conceptual Framework for Learning in Control (2010, methodology): “GPs for system identification and robust model predictive control have been employed for example by Kocijan et al. (2004), Murray-Smith et al. (2003), Murray-Smith and Sbarbaro (2002), Grancharova et al.”
A Survey on Policy Search for Robotics (2013, methodology): “These variances can even lead to slower convergence than gradient-free policy updates [7, 35].”
Learning Non-stationary System Dynamics Online Using Gaussian Processes (2010, methodology): “[6] presented an approach to improve a motion model of a blimp derived from aeronautic principles by using a GP to model the residual.”
Reducing hardware experiments for model learning and policy optimization (2015, methodology): “Ko et al. [12] used Gaussian Process Simulation Hardware Policy Dynamics Bias Learning Policy Search results update control input update cost Fig.”
Model-based contextual policy search for data-efficient generalization of robot skills (2017, methodology): “We refer to such algorithms as model-based policy search algorithms [10, 1, 4, 34, 2, 18, 29].”
Autonomous blimp control using model-free reinforcement learning in a continuous state and action space (2007, methodology): “The approach that is most closely related to the one described in this paper has recently been presented by Ko et al. [11].”
PILCO: A Model-Based and Data-Efficient Approach to Policy Search (2011, methodology): “Algorithms with GP dynamics models in RL were presented by Rasmussen & Kuss (2004), Ko et al. (2007), and Deisenroth et al. (2009).”