Exploiting Model Uncertainty Estimates for Safe Dynamic Control Learning

By Jeff Schneider. 1996

Subfields: Reinforcement Learning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Safety, Robustness & Interpretability

Citations in the corpus

From Pixels to Torques: Policy Learning with Deep Dynamical Models (2015, methodology): “ata is collected in unexplored regions, including the region that we actually are interested in. There are two solutions to this problem: Either we use a probabilistic dynamics model as suggested in (Schneider, 1997; Deisenroth et al., 2015) to explicitly account for model uncertainty and the implie”
Reinforcement learning in robotics: A survey (2013, methodology): “To apply reinforcement learning in robotics, safe exploration becomes a key issue of the learning process (Schneider, 1996; Bagnell, 2004; Deisenroth and Rasmussen, 2011; Moldovan and Abbeel, 2012), a problem often neglected in the general reinforcement learning community.”
Uncertainty-Aware Reinforcement Learning for Collision Avoidance (2017, background): “Several model-based robotic learning algorithms have been proposed that explicitly reason about uncertainty [3, 26].”
Model-Ensemble Trust-Region Policy Optimization (2018, background): “Previous work has pointed out this issue as model bias , i.e. (Deisenroth & Rasmussen, 2011; Schneider, 1997; Atkeson & Santamaria, 1997).”
An Algorithmic Perspective on Imitation Learning (2018, methodology): “Recently, Schulman et al. [2013] proposed a method to generalize the demonstrated trajectories based on non-rigid registration.”
Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning (2011, background): “Reducing model bias requires accounting for model uncertainty during planning [23].”
Autonomous helicopter control using reinforcement learning policy search methods (2001, methodology): “It has previously been demonstrated that this criterion (or rather the approximation of it given in [4] ) typically leads to controllers that are neither too conservative, nor as aggressive as that obtained by a optimizing a maximum likelihood model.”
Gaussian Processes for Data-Efficient Learning in Robotics and Control (2015, critical): “The main reason why model-based methods are not widely used in RL is that they can suffer severely from model errors , i.e., they inherently assume that the learned model resembles the real environment sufﬁciently accurately [5], [48], [49].”
Learning deep dynamical models from image pixels (2014, background): “In a setting where we make decisions based on (onestep or multiple-step ahead) predictions, such as optimal control or model-based reinforcement learning, a probabilistic model is often needed for robust decision making as we need to account for models errors [13, 3].”
Meta Reinforcement Learning with Latent Variable Gaussian Processes (2018, methodology): “In this context, [27, 10] highlighted that probabilistic models of f are essential for data-efﬁcient learning as they mitigate the effect of model errors.”
Model-based imitation learning by probabilistic trajectory matching (2013, background): “The GP takes uncertainties about the learned robot dynamics into account and reduces typical problems of learned models, such as model errors [25], [3] and potentially unrealistic assumptions (e.”
Data-Efficient Generalization of Robot Skills with Contextual Policy Search (2013, methodology): “One solution to alleviate this problem is to represent the uncertainty of the learned model itself by using probabilistic models (Schneider 1997; Deisenroth et al. 2011).”
A Survey of Deep Learning Applications to Autonomous Vehicle Control (2019, background): “Exploration poses safety challenges as the agent is encouraged to take random actions, which can lead to catastrophic events if not considered beforehand [210]–[213].”
Sample-based informationl-theoretic stochastic optimal control (2014, background): “In this case, our bound provides us with the additional benefit that we avoid exploiting the possibly inaccurate model greedily.”
Data-Efficient Learning of Feedback Policies from Image Pixels using Deep Dynamical Models (2015, methodology): “The quality of the NMPC controller is inherently bound to the prediction quality of the dynamical model, which is typical in model-based RL [5, 14, 29].”