Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Tasks

By Bohan Wu, Suraj Nair, Li Fei-Fei, Chelsea Finn. 2021

26 citations recorded by the source corpus.

In this paper, we study the problem of learning a repertoire of low-level skills from raw images that can be sequenced to complete long-horizon visuomotor tasks. Reinforcement learning (RL) is a promising approach for acquiring short-horizon skills autonomously. However, the focus of RL algorithms has largely been on the success of those individual skills, more so than learning and grounding a large repertoire of skills that can be sequenced to complete extended multi-stage tasks. The latter demands robustness and persistence, as errors in skills can compound over time, and may require the robot to have a number of primitive skills in its repertoire, rather than just one. To this end, we introduce EMBER, a model-based RL method for learning primitive skills that are suitable for completing long-horizon visuomotor tasks. EMBER learns and plans using a learned model, critic, and success classifier, where the success classifier serves both as a reward function for RL and as a grounding mechanism to continuously detect if the robot should retry a skill when unsuccessful or under perturbations. Further, the learned model is task-agnostic and trained using data from all skills, enabling the robot to efficiently learn a number of distinct primitives. These visuomotor primitive skills and their associated pre- and post-conditions can then be directly combined with off-the-shelf symbolic planners to complete long-horizon tasks. On a Franka Emika robot arm, we find that EMBER enables the robot to complete three long-horizon visuomotor tasks at 85% success rate, such as organizing an office desk, a file cabinet, and drawers, which require sequencing up to 12 skills, involve 14 unique learned primitives, and demand generalization to novel objects.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Control and Systems Engineering, Mechanical Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Sergey Levine
  2. Chelsea Finn
  3. Pieter Abbeel
  4. Silvio Savarese
  5. Li Fei-Fei
  6. Quoc V. Le
  7. Yoshua Bengio
  8. Trevor Darrell

In Computer Vision and Pattern Recognition

  1. Sergey Levine
  2. Luc Van Gool
  3. Jitendra Malik
  4. Li Fei-Fei
  5. Dacheng Tao
  6. Trevor Darrell
  7. Andrew Zisserman
  8. Chelsea Finn

In Control and Systems Engineering

  1. Sergey Levine
  2. Chelsea Finn
  3. Pieter Abbeel
  4. Karol Hausman
  5. Silvio Savarese
  6. Pierre Sermanet
  7. Suraj Nair
  8. Julian Ibarz

In Mechanical Engineering

  1. Sergey Levine
  2. Chelsea Finn
  3. Brian Ichter
  4. Pierre Sermanet
  5. Suraj Nair
  6. Ted Xiao
  7. Peter Pastor
  8. Fei Xia

In Reinforcement Learning

  1. Sergey Levine
  2. Chelsea Finn
  3. Pieter Abbeel
  4. Karol Hausman
  5. Suraj Nair
  6. Pierre Sermanet
  7. Silvio Savarese
  8. Ted Xiao

Citations in the corpus