Infinite-Horizon Model Predictive Control for Periodic Tasks with Contacts

By Tom Erez, Yuval Tassa, E. Todorov. 2011

112 citations recorded by the source corpus.

We present a method that combines offline trajectory optimization and online Model Predictive Control (MPC), generating robust controllers for complex periodic behavior in domains with unilateral constraints (e.g., contact with the environment). MPC offers robust and adaptive control even in high-dimensional domains; however, the online optimization gets stuck in local minima when the domains has discontinuous dynamics. Some methods of trajectory optimization that are immune to such problems, but these are often too slow to be applied online. In this paper, we use offline optimization to find the limit-cycle solution of an infinite-horizon average-cost optimal-control task. We then compute a local quadratic approximation of the Value function around this limit cycle. Finally, we use this quadratic approximation as the terminal cost of an online MPC. This combination of an offline solution of the infinite-horizon problem with an online MPC controller is known as Infinite Horizon Model Predictive Control (IHMPC), and has previously been applied only to simple stabilization objectives. Here we extend IHMPC to tackle periodic tasks, and demonstrate the power of our approach by synthesizing hopping behavior in a simulated robot. IHMPC involves a limited computational load, and can be executed online on a standard laptop computer. The resulting behavior is extremely robust, allowing the hopper to recover from virtually any perturbation. In real robotic domains, modeling errors are inevitable. We show how IHMPC is robust to modeling errors by altering the morphology of the robot; the same controller remains effective, even when the underlying infinite-horizon solution is no longer accurate.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

Benchmarking Deep Reinforcement Learning for Continuous Control (2016, neutral): “…dynamics and difficulty: Swimmer (Purcell, 1977; Coulom, 2002; Levine & Koltun, 2013; Schulman et al., 2015a), Hopper (Murthy & Raibert, 1984; Erez et al., 2011; Levine & Koltun, 2013; Schulman et al., 2015a), Walker (Raibert & Hodgins, 1991; Erez et al., 2011; Levine & Koltun, 2013;…”
Towards Generalization and Simplicity in Continuous Control (2017, background): “Finding global policies on these tasks with on-line model based optimization has been a challenging benchmark in literature due to under-actuation, high dimensionality, and presence of contacts (Erez et al., 2011).”
EPOpt: Learning Robust Neural Network Policies Using Model Ensembles (2016, methodology): “Descriptions of these tasks are below: Hopper: The hopper task is to make a 2D planar hopper with three joints and 4 body parts hop forward as fast as possible [38].”
Deep Haptic Model Predictive Control for Robot-Assisted Dressing (2017, background): “Some examples include aerial control vehicles [11], [12] and robot locomotion [13], [14].”