Self-supervised 6D Object Pose Estimation for Robot Manipulation

By Xinke Deng, Yu Xiang, Arsalan Mousavian, Clemens Eppner, Timothy Bretl, Dieter Fox. 2019

231 citations recorded by the source corpus.

To teach robots skills, it is crucial to obtain data with supervision. Since annotating real world data is timeconsuming and expensive, enabling robots to learn in a self- supervised way is important. In this work, we introduce a robot system for self-supervised 6D object pose estimation. Starting from modules trained in simulation, our system is able to label real world images with accurate 6D object poses for self-supervised learning. In addition, the robot interacts with objects in the environment to change the object configuration by grasping or pushing objects. In this way, our system is able to continuously collect data and improve its pose estimation modules. We show that the self-supervised learning improves object segmentation and 6D pose estimation performance, and consequently enables the system to grasp objects more reliably. A video showing the experiments can be found at https://youtu.be/W1Y0Mmh1Gd8.

Subfields: Aerospace Engineering, Biomedical Engineering, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Biomedical Engineering

In Control and Systems Engineering

Citations in the corpus

Transporter Networks: Rearranging the Visual World for Robotic Manipulation (2020, background): “Considerable research in vision for manipulation has been devoted to object detectors and pose estimators [17, 18, 19, 20, 21].”
CLIPort: What and Where Pathways for Robotic Manipulation (2021, background): “Traditionally, perception for manipulation has centered around object detectors, segmentors, and pose estimators [21, 22, 23, 24, 25, 26].”
Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes (2021, background): “Model-based grasping circumvents reasoning about the physics of contact and grasp generation by pre-defining a set of grasps in the object frame and transform those grasps according to the 6-DoF object pose [1, 2, 3, 4] or detected keypoints of the objects [5, 6].”
GraspNet-1Billion: A Large-Scale Benchmark for General Object Grasping (2020, background): “With the rapid development in monocular object 6D pose estimation [17, 45], some researchers [8] predict 6D poses of the objects and project predefined grasp poses to the scene.”
Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation (2022, methodology): “Traditionally, methods in robot perception have used explicit “object” representations like instance segmentations, object classes, poses [18, 19, 20, 21, 22, 23].”
Few-Shot Object Detection and Viewpoint Estimation for Objects in the Wild (2020, background): “Some work also focuses on training the viewpoint estimation network on a collection of unlabeled images by self-supervised learning [69], [70], [71].”