Exploring Deep and Recurrent Architectures for Optimal Control

By Sergey Levine. 2013

24 citations recorded by the source corpus.

Sophisticated multilayer neural networks have achieved state of the art results on multiple supervised tasks. However, successful applications of such multilayer networks to control have so far been limited largely to the perception portion of the control pipeline. In this paper, we explore the application of deep and recurrent neural networks to a continuous, high-dimensional locomotion task, where the network is used to represent a control policy that maps the state of the system (represented by joint angles) directly to the torques at each joint. By using a recent reinforcement learning algorithm called guided policy search, we can successfully train neural network controllers with thousands of parameters, allowing us to compare a variety of architectures. We discuss the differences between the locomotion control task and previous supervised perception tasks, present experimental results comparing various architectures, and discuss future directions in the application of techniques from deep learning to the problem of optimal control.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

Citations in the corpus

Learning contact-rich manipulation skills with guided policy search (2015, background): “Unlike in our previous work [30], we found that a deeper two-layer architecture was necessary to capture the complexity of the manipulation behaviors.”
Memory Augmented Control Networks (2017, background): “Using recurrent networks to store past information and learn optimal control has been explored before in [9].”
The importance of experience replay database composition in deep reinforcement learning (2015, background): “However, the regularization of neural networks used for control is currently not as well understood as the regularization of neural networks in other domains [17].”
Learning Complex Neural Network Policies with Trajectory Optimization (2014, methodology): “We omitted the foot contact and forward kinematics features that we used in prior work (Levine, 2013), and instead trained policies that map the state vector directly to joint torques.”