Supervision via competition: Robot adversaries for learning tasks

By Lerrel Pinto, James Davidson, Abhinav Kumar Gupta. 2016

85 citations recorded by the source corpus.

There has been a recent paradigm shift in robotics to data-driven learning for planning and control. Due to large number of experiences required for training, most of these approaches use a self-supervised paradigm: using sensors to measure success/failure. However, in most cases, these sensors provide weak supervision at best. In this work, we propose an adversarial learning framework that pits an adversary against the robot learning the task. In an effort to defeat the adversary, the original robot learns to perform the task with more robustness leading to overall improved performance. We show that this adversarial framework forces the robot to learn a better grasping model in order to overcome the adversary. By grasping 82% of presented novel objects compared to 68% without an adversary, we demonstrate the utility of creating adversaries. We also demonstrate via experiments that having robots in adversarial setting might be a better learning strategy as compared to having collaborative multiple robots. For supplementary video see: youtu.be/QfK3Bqhc6Sk

Subfields: Artificial Intelligence, Control and Systems Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Reinforcement Learning

Citations in the corpus

Multi-task Self-Supervised Visual Learning (2017, background): “In the last few years, a considerable number of such tasks have been proposed [1, 2, 6, 7, 8, 17, 20, 21, 23, 25, 26, 27, 28, 29, 31, 39, 40, 42, 43, 46, 47], such as asking a neural network to colorize grayscale images, ﬁll in image holes, solve jigsaw puzzles made from image patches, or predict…”
Towards the Science of Security and Privacy in Machine Learning (2016, background): “Pinto et al. improve a model for grasping objects by introducing a competing model that attempts to snatch objects before the original model successfully grasps them [69].”
Solving Rubik's Cube with a Robot Hand (2019, methodology): “[83] propose to use adversarial training to obtain more robust policies and show that it also helps with transfer to physical robots [82].”
Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation (2018, background): “This has been followed by attempts to scale even further using multiple robots [3], multiple tasks [19], adversarial learning [20] and curriculum learning [21].”
Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments (2017, background): “Additionally, the competitive component of the environment makes it not only nonstationary but also adversarial, which provides a natural training curriculum and encourages learning robust strategies [7, 19, 20]. We evaluate our meta-learning agents along with a number of baselines on a (single-agen”
Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics (2017, background): “recently Pinto and Gupta [43] scaled up data collection by recording over 40k grasping experiences on a Baxter and iteratively training CNNs to predict lifting successes or to resist grasp perturbations caused by an adversary [44].”
A-Fast-RCNN: Hard Positive Generation via Adversary for Object Detection (2017, background): “Our work is also related to a recent work on adversarial training in robotics [25].”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, unknown): “A novel extension of pixel-level domain adaptation named GraspGAN [237] was proposed for robotic grasping [238], [239].”
Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching (2017, methodology): “It is also possible to exploit local features of objects without object identity to efficiently detect grasps (Gualtieri et al., 2017; Lenz et al., 2015; Levine et al., 2018; Mahler et al., 2017; Morales et al., 2004; Pinto et al., 2017; Pinto and Gupta, 2016; Redmon and Angelova, 2015; ten Pas and ”
Adversarial Transfer Learning (2018, background): “[99], the adversary tries to learn to behave (either through shaking or snatching) such that the robot cannot grasp the object.”
Robust Adversarial Reinforcement Learning (2017, background): “In (Pinto et al., 2016) two supervised agents were trained with one acting as an adversary for selfsupervised learning which showed improved robot grasping.”
GPLAC: Generalizing Vision-Based Robotic Skills Using Weakly Labeled Images (2017, background): “…the success of deep learning meth-ods has prompted considerable recent interest in learning sensorimotor policies that directly map image pixels to actions [52, 27, 1, 29, 45, 43, 24, 37, 38], following on classic work on end-to-end imitation and reinforcement learning from the 1980s and 90s [40].”
Domain Randomization and Generative Models for Robotic Grasping (2017, background): “To alleviate this problem, some work explores automated large-scale data collection [23], [36].”
Learning Dexterous In-Hand Manipulation. (2018, background): “[48] propose to use adversarial training to obtain more robust policies and show that it also helps with transfer to physical robots [47].”
Learning Visual Affordances for Robotic Manipulation (2019, methodology): “This is inspired by recent data-driven methods for grasp planning [77, 102, 59, 93, 86, 87, 66, 37, 61], which do not rely on object identities or state estimation.”