A Framework for Data-Driven Robotics

By Serkan Cabi, Sergio Gomez Colmenarejo, Alexander Novikov, Ksenia Konyushkova, Scott E. Reed, Rae Jeong, Konrad Zolna, Yusuf Aytar, David M. Budden, Mel Vecerík and 6 more. 2019

30 citations recorded by the source corpus.

We present a framework for data-driven robotics that makes use of a large dataset of recorded robot experience and scales to several tasks using learned reward functions. We show how to apply this framework to accomplish three different object manipulation tasks on a real robot platform. Given demonstrations of a task together with task-agnostic recorded experience, we use a special form of human annotation as supervision to learn a reward function, which enables us to deal with real-world tasks where the reward signal cannot be acquired directly. Learned rewards are used in combination with a large dataset of experience from different tasks to learn a robot policy offline using batch RL. We show that using our approach it is possible to train agents to perform a variety of challenging manipulation tasks including stacking rigid objects and handling cloth.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Control and Systems Engineering, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Sergey Levine
2. Ofir Nachum
3. Chelsea Finn
4. Pieter Abbeel
5. George Tucker
6. Aviral Kumar
7. Yoshua Bengio
8. Ruslan Salakhutdinov

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Jitendra Malik
3. Dacheng Tao
4. Sergey Levine
5. Trevor Darrell
6. Xiaogang Wang
7. Li Fei-Fei
8. Andrew Zisserman

In Control and Systems Engineering

1. Sergey Levine
2. Justin Fu
3. Aviral Kumar
4. George Tucker
5. Nando de Freitas
6. Ziyun Wang
7. Danfei Xu
8. Mohammad Norouzi

In Robotics & Control

1. Aviral Kumar
2. Justin Fu
3. George Tucker
4. Danfei Xu
5. Nando de Freitas
6. Ziyun Wang
7. Jonathan Kelly
8. Mohammad Norouzi

Citations in the corpus

• Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020, methodology): “Cabi et al. (2019) propose to use reward learning from human preferences combined with offline RL to provide a user-friendly method for controlling robots for object manipulation tasks.”
• Hyperparameter Selection for Offline Reinforcement Learning (2020, background): “These methods are useful when it is challenging, risky or expensive to have an arbitrary policy interact with an environment, for example for robotics (Cabi et al., 2019), self-driving cars, health care (Futoma et al., 2020), or dialogue (Jaques et al., 2019).”
• An Optimistic Perspective on Offline Reinforcement Learning (2019, background): “Recently, Cabi et al. (2019) also show success of standard RL algorithms in the offline setting on large-scale robotic datasets.”
• COG: Connecting New Skills to Past Experience with Offline Reinforcement Learning (2020, unknown): “Our paper is more closely related to Kalashnikov et al. [11], Julian et al. [22], and Cabi et al. [23], which also use offline RL and large prior datasets.”
• Positive-Unlabeled Reward Learning (2019, methodology): “Recent works have made promising progress in learning reward functions directly from human supervision, such as ratings (Cabi et al., 2019) and behavior preferences (Wilson et al.”
• Fighting Failures with FIRE: Failure Identification to Reduce Expert Burden in Intervention-Based Learning (2020, result): “This approach has been empirically shown to have varying degrees of success [8], [15], [18], [19].”