An Online Learning Approach to Model Predictive Control

By Nolan Wagener, Ching-An Cheng, Jacob Sacks, Byron Boots. 2019

84 citations recorded by the source corpus.

Model predictive control (MPC) is a powerful technique for solving dynamic control tasks. In this paper, we show that there exists a close connection between MPC and online learning, an abstract theoretical framework for analyzing online decision making in the optimization literature. This new perspective provides a foundation for leveraging powerful online learning algorithms to design MPC algorithms. Specifically, we propose a new algorithm based on dynamic mirror descent (DMD), an online learning algorithm that is designed for non-stationary setups. Our algorithm, Dynamic Mirror Descent Model Predictive Control (DMD-MPC), represents a general family of MPC algorithms that includes many existing techniques as special instances. DMD-MPC also provides a fresh perspective on previous heuristics used in MPC and suggests a principled way to design new MPC algorithms. In the experimental section of this paper, we demonstrate the flexibility of DMD-MPC, presenting a set of new MPC algorithms on a simple simulated cartpole and a simulated and real-world aggressive driving task. Videos of the real-world experiments can be found at this https URL and this https URL.

Subfields: Control and Systems Engineering, Management Science and Operations Research, Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

1. Martha White
2. Gabriel Dulac-Arnold
3. Philip S. Thomas
4. Georgios Theocharous
5. Sridhar Mahadevan
6. Arthur Argenson
7. Yash Chandak
8. Alexander Liniger

In Management Science and Operations Research

1. Philip S. Thomas
2. Georgios Theocharous
3. Matteo Hessel
4. Hado van Hasselt
5. John Aslanides
6. Gabriel Dulac-Arnold
7. Yash Chandak
8. Martha White

In Reinforcement Learning

1. Sergey Levine
2. Matteo Hessel
3. Philip S. Thomas
4. Pieter Abbeel
5. Martha White
6. Hado van Hasselt
7. John Aslanides
8. Ofir Nachum

In Robotics & Control

1. Gabriel Dulac-Arnold
2. Philip S. Thomas
3. Georgios Theocharous
4. Sridhar Mahadevan
5. Arthur Argenson
6. Yash Chandak
7. Alexander Liniger
8. Matteo Hessel

Citations in the corpus

• When to use parametric models in reinforcement learning? (2019, methodology): “This is commonly and successfully employed in model-predictive control (Richalet et al., 1978; Morari and Lee, 1999; Mayne, 2014; Wagener et al., 2019).”
• Optimizing for the Future in Non-Stationary MDPs (2020, background): “Other works have also shown how models of the environment can be used for continual learning (Lu et al., 2019) or using it along with a model predictive control (Wagener et al., 2019).”
• Model-Based Offline Planning (2020, background): “The β parameter can be interpreted as a form of learning rate defining how quickly the current optimal trajectory should change with new rollout information (Wagener et al., 2019).”
• Autonomous Vehicles on the Edge: A Survey on Autonomous Vehicle Racing (2022, background): “[197] 2019 Learning Based Control Dynamic Mirror Descent MPC (DMD-MPC) No AutoRally Kabzan et al.”