Self-Supervised Learning for Precise Pick-and-Place Without Object Model
By Lars Berscheid, Pascal Meissner, Torsten Kröger. 2020
72 citations recorded by the source corpus.
Flexible pick-and-place is a fundamental yet challenging task within robotics, in particular due to the need of an object model for a simple target pose definition. In this work, the robot instead learns to pick-and-place objects using planar manipulation according to a single, demonstrated goal state. Our primary contribution lies within combining robot learning of primitives, commonly estimated by fully-convolutional neural networks, with one-shot imitation learning. Therefore, we define the place reward as a contrastive loss between real-world measurements and a task-specific noise distribution. Furthermore, we design our system to learn in a self-supervised manner, enabling real-world experiments with up to 25 000 pick-and-place actions. Then, our robot is able to place trained objects with an average placement error of (2.7 <inline-formula><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 0.2) mm and (2.6 <inline-formula><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 0.8)<inline-formula><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula>. As our approach does not require an object model, the robot is able to generalize to unknown objects while keeping a precision of (5.9 <inline-formula><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 1.1) mm and (4.1 <inline-formula><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 1.2)<inline-formula><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula>. We further show a range of emerging behaviors: The robot naturally learns to select the correct object in the presence of multiple object types, precisely inserts objects within a peg game, picks screws out of dense clutter, and infers multiple pick-and-place actions from a single goal state.
Subfields: Artificial Intelligence, Biomedical Engineering, Control and Systems Engineering
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Li Fei-Fei
- Roozbeh Mottaghi
- Josh Tenenbaum
- Aniruddha Kembhavi
- Yuke Zhu
- Mohit Bansal
- Abhinav Kumar Gupta
In Biomedical Engineering
- Andy Zeng
- Johnny Lee
- Peter R. Florence
- Yuke Zhu
- Li Fei-Fei
- Alberto Rodriguez
- Yilun Du
- Josh Tenenbaum
In Control and Systems Engineering
- Li Fei-Fei
- Andy Zeng
- Yuke Zhu
- Johnny Lee
- Linxi (Jim) Fan
- Peter R. Florence
- Anima Anandkumar
- Jonathan Tompson
Citations in the corpus
- Transporter Networks: Rearranging the Visual World for Robotic Manipulation (2020, background): “, logistics, agriculture, household) – leading to renewed interest in leveraging learned models to achieve general pick-and-place policies [23, 24, 25, 26, 27, 28, 29] that can handle unseen objects.”
- Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation (2021, background): “Direct learning of pick-and-place policies meanwhile requires large amounts of data from demonstrations [2, 14, 40].”
- VIMA: General Robot Manipulation with Multimodal Prompts (2022, background): “While “pick-and-place” and “wipe” seem simple, they do cover a wide range of tabletop manipulation tasks and are crucial to industrial use cases like warehouse robots (Yoon et al., 2003; Berscheid et al., 2020; Devin et al., 2020; Song et al., 2019).”
- Visual Room Rearrangement (2021, background): “Some works address some of these issues, such as generalization to new objects or imperfect perception [54, 6].”