Information theoretic MPC for model-based reinforcement learning

By Grady Williams, Nolan Wagener, Brian Goldfain, Paul Drews, James M. Rehg, Byron Boots, Evangelos A. Theodorou. 2017

671 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Control and Systems Engineering, Reinforcement Learning, Information Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Reinforcement Learning

In Information Theory

Citations in the corpus

Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models (2018, methodology): “…in parametric function approximators, such as NNs, suggest that such methods are worth revisiting [Baranes and Oudeyer, 2013, Fu et al., 2016, Punjani and Abbeel, 2015, Lenz et al., 2015, Agrawal et al., 2016, Gal et al., 2016, Depeweg et al., 2016, Williams et al., 2017, Nagabandi et al., 2017].”
MOReL : Model-Based Offline Reinforcement Learning (2020, methodology): “Furthermore, MBRL can draw upon the rich literature on model-based planning including model predictive control (MPC) [23, 24, 61, 62], search based planning [25, 63], dynamic programming [26, 64], and policy optimization [32, 65, 33, 27, 53].”
Agile Autonomous Driving using End-to-End Deep Imitation Learning (2017, background): “In real-world experiments, we show the AutoRally car—without any state estimator or online planning, but with a DNN policy that directly inputs measurements from a low-cost monocular camera and wheel speed sensors—could learn to perform high-speed driving at an average speed of ∼6 m/s and a top spee”
Deep Dynamics Models for Learning Dexterous Manipulation (2019, background): “Other work has focused on learning these models using high-capacity function approximators [17, 18, 3, 19, 20, 21, 22] and probabilistic dynamics models [23, 24, 25, 26].”
Learning to Adapt in Dynamic, Real-World Environments through Meta-Reinforcement Learning (2018, methodology): “In contrast, model-based methods attain superior sample efﬁciency by ﬁrst learning a model of system dynamics, and then using that model to optimize a policy (Deisenroth et al., 2013; Lenz et al., 2015; Levine et al., 2016; Nagabandi et al., 2017b; Williams et al., 2017).”
Robustness via Retrying: Closed-Loop Robotic Manipulation with Self-Supervised Learning (2018, background): “Model-predictive control (MPC) [24] has proven successful in a variety of robotic tasks [25, 26, 27, 28, 29].”
A Game Theoretic Framework for Model Based Reinforcement Learning (2020, methodology): “…have used data augmentation and proposed additional components to enhance efficiency and stability, such as the use of model predictive control (Williams et al., 2017; Lowrey et al., 2019; Nagabandi et al., 2019), uncertainty quantification through Bayesian models (Deisenroth & Rasmussen,…”
An Online Learning Approach to Model Predictive Control (2019, background): “which reduces to the MPPI update rule [31] for γt = 1.”
Intelligent Autonomous Things on the Battlefield (2019, neutral): “2011; Williams et. al.”
Learning to Adapt: Meta-Learning for Model-Based Control (2018, background): “This use of MPC allows us to better deal with the imperfections of our dynamics model, because we replan at each time step using updated state information.”
Aggressive Deep Driving: Model Predictive Control with a CNN Cost Model (2017, methodology): “In this paper we only use an instantaneous running cost (there is no terminal cost), and we sample trajectories on a GPU using the dynamics model from [4].”
Constrained Exploration and Recovery from Experience Shaping (2018, methodology): “For such critical applications, deep neural networks can be used as a specialized service for components offering stability and performance guarantees, e.g., for visual recognition or dynamics prediction in conjunction with model-predictive control (Williams et al. 2017).”
Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (2024, background): “Success has also been achieved in high-speed, off-road driving with model-based RL (90) and, recently, vision-based model-free RL (91).”
Aggressive Deep Driving: Combining Convolutional Neural Networks and Model Predictive Control (2017, methodology): “In this paper we only use an instantaneous running cost (there is no terminal cost), and we sample trajectories on a GPU using the dynamics model from [4].”
Autonomous Vehicles on the Edge: A Survey on Autonomous Vehicle Racing (2022, methodology): “The research published with the AutoRally car is in the field of planning [11], [115], [204], [215], [216] and control [50], [62], [114], [197], [203], [205]–[209], state estimation [56], [214] and the application of deep neural networks for perception and”
Information-Theoretic Model Predictive Control: Theory and Applications to Autonomous Driving (2017, methodology): “In terms of technical contributions, we present a modified version of the algorithm presented in [22], which is more flexible and takes into account practical considerations, such as smoothing.”
MultiNet: Multi-Modal Multi-Task Learning for Autonomous Driving (2017, methodology): “As part of our research on autonomous driving under unstructured conditions, such as sidewalks, unpaved roads, and trails, we collected driving data using a fleet of 1/10th scale model cars (Figure 2) similar to that of [6, 24, 11, 17, 18, 16].”