End-to-End Training of Deep Visuomotor Policies

By Sergey Levine, Chelsea Finn, Trevor Darrell, Pieter Abbeel. 2015

3,853 citations recorded by the source corpus.

Policy search methods can allow robots to learn control policies for a wide range of tasks, but practical applications of policy search often require hand-engineered components for perception, state estimation, and low-level control. In this paper, we aim to answer the following question: does training the perception and control systems jointly end-to-end provide better performance than training each component separately? To this end, we develop a method that can be used to learn policies that map raw image observations directly to torques at the robot's motors. The policies are represented by deep convolutional neural networks (CNNs) with 92,000 parameters, and are trained using a partially observed guided policy search method, which transforms policy search into supervised learning, with supervision provided by a simple trajectory-centric reinforcement learning method. We evaluate our method on a range of real-world manipulation tasks that require close coordination between vision and control, such as screwing a cap onto a bottle, and present simulated comparisons to a range of prior policy search methods.

Subfields: Aerospace Engineering, Artificial Intelligence, Control and Systems Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

  1. Sergey Levine
  2. Jitendra Malik
  3. Abhinav Kumar Gupta
  4. Silvio Savarese
  5. Kristen Grauman
  6. Pieter Abbeel
  7. Dieter Fox
  8. Vladlen Koltun

In Artificial Intelligence

  1. Sergey Levine
  2. Pieter Abbeel
  3. Yoshua Bengio
  4. Chelsea Finn
  5. Mohit Bansal
  6. Ruslan Salakhutdinov
  7. Trevor Darrell
  8. Quoc V. Le

In Control and Systems Engineering

  1. Sergey Levine
  2. Pieter Abbeel
  3. Chelsea Finn
  4. Jan Peters
  5. Vikash Kumar
  6. Abhishek Gupta
  7. Dieter Fox
  8. Abhinav Kumar Gupta

In Reinforcement Learning

  1. Sergey Levine
  2. Pieter Abbeel
  3. Chelsea Finn
  4. Ofir Nachum
  5. Shimon Whiteson
  6. Shixiang Shane Gu
  7. Yuxi Li
  8. Abhishek Gupta

Citations in the corpus