PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
By Yu Xiang, Tanner Schmidt, Venkatraman Narayanan, Dieter Fox. 2017
2,347 citations recorded by the source corpus.
Estimating the 6D pose of known objects is important for robots to interact with the real world. The problem is challenging due to the variety of objects as well as the complexity of a scene caused by clutter and occlusions between objects. In this work, we introduce PoseCNN, a new Convolutional Neural Network for 6D object pose estimation. PoseCNN estimates the 3D translation of an object by localizing its center in the image and predicting its distance from the camera. The 3D rotation of the object is estimated by regressing to a quaternion representation. We also introduce a novel loss function that enables PoseCNN to handle symmetric objects. In addition, we contribute a large scale video dataset for 6D object pose estimation named the YCB-Video dataset. Our dataset provides accurate 6D poses of 21 objects from the YCB dataset observed in 92 videos with 133,827 frames. We conduct extensive experiments on our YCB-Video dataset and the OccludedLINEMOD dataset to show that PoseCNN is highly robust to occlusions, can handle symmetric objects, and provide accurate pose estimation using only color images as input. When using depth data to further refine the poses, our approach achieves state-of-the-art results on the challenging OccludedLINEMOD dataset. Our code and dataset are available at this https URL.
Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, 3D Vision & Pose, Human-Computer Interaction, Robotics & Control
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Jitendra Malik
- Dacheng Tao
- Trevor Darrell
- Xiaogang Wang
- Ming-Hsuan Yang
- Cordelia Schmid
- Andrew Zisserman
In Control and Systems Engineering
- Sergey Levine
- Dieter Fox
- Chelsea Finn
- Pieter Abbeel
- Silvio Savarese
- Li Fei-Fei
- Jitendra Malik
- Yuke Zhu
In 3D Vision & Pose
- Jitendra Malik
- Abhinav Kumar Gupta
- Silvio Savarese
- Shubham Tulsiani
- Leonidas J. Guibas
- Angjoo Kanazawa
- David F. Fouhey
- Andreas Geiger
In Human-Computer Interaction
- Cordelia Schmid
- Jitendra Malik
- Bugra Tekin
- Pascal Fua
- Francesc Moreno-Noguer
- Angjoo Kanazawa
- Dieter Fox
- Xiaowei Zhou
In Robotics & Control
- Sergey Levine
- Chelsea Finn
- Dieter Fox
- Silvio Savarese
- Abhinav Kumar Gupta
- Li Fei-Fei
- Pieter Abbeel
- Yuke Zhu
Citations in the corpus
- Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applications (2022, background): “One highly related direction is to model and estimate 3D hand joints [5,44,45,53,60,63,66,72,78,79,80,85,3] and mesh [26,27,51,55,75,83,46], object pose [14,28,29,35,43,57,64,71,76,81], or both [6,17,18,19,69].”
- Continuous Scene Representations for Embodied AI (2022, methodology): “To demonstrate the potential of the representation on real-world data, we additionally evaluate CSR features, trained in simulation on object tracking in YCB-Video [59].”
- RLBench: The Robot Learning Benchmark & Learning Environment (2019, background): “Moving on from whole-system benchmarks, there are a host of benchmarks that focus on sub-problems, for example perception datasets, from both the computer vision community (such as ILSVRC [1], COCO [30], Pascal-VOC [31], etc), and the robotics community (such as BigBIRD [32], YCB-Video [33], etc).”
- One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning (2018, background): “Second, learning from raw visual inputs typically requires a substantial amount of data, with modern deep learning vision systems using hundreds of thousands to millions of images [57, 18].”
- CLIPort: What and Where Pathways for Robotic Manipulation (2021, background): “Traditionally, perception for manipulation has centered around object detectors, segmentors, and pose estimators [21, 22, 23, 24, 25, 26].”
- HOnnotate: A Method for 3D Annotation of Hand and Object Poses (2019, methodology): “We use objects from the YCB-Video dataset [68], as they have various shapes and materials, and can be bought online [ 1] by researchers interested in performing their own experiments.”
- PVN3D: A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation (2019, methodology): “The ADD-S metric is designed for symmetric objects and the mean distance is computed based on the closest point distance: ) For evaluation, we follow [52, 50] and compute the ADD-S AUC, the area under the accuracy-threshold curve, which is obtained by varying the distance threshold in evaluation.”
- DPOD: 6D Pose Object Detector and Refiner (2019, methodology): “We demonstrated that for both, real and synthetic training data, our detector outperforms other related works, such as [32, 33], by a large margin and performs similarly to [25].”
- Articulation-Aware Canonical Surface Mapping (2020, background): “Towards this general goal of learning representations that respect correspondence, several prior works attempt to design [22], or learn features invariant to camera movement [9, 38], or synthetic transforms [30].”
- Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects (2018, methodology): “While deep neural networks have been successfully applied to the problem of object detection in 2D [1, 2, 3], they have only recently begun to be applied to 3D object detection and pose estimation [4, 5, 6].”
- Leveraging Photometric Consistency Over Time for Sparsely Supervised Hand-Object Reconstruction (2020, positive): “As in [22, 47], we predict values that can be easily estimated from image evidence.”
- Unseen Object Instance Segmentation for Robotic Environments (2020, methodology): “Many methods employ some level of rendering randomization [59], [60], [61], [9], [62], [63], including lighting conditions and textures.”
- Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation (2022, methodology): “Traditionally, methods in robot perception have used explicit “object” representations like instance segmentations, object classes, poses [18, 19, 20, 21, 22, 23].”
- GraspNet-1Billion: A Large-Scale Benchmark for General Object Grasping (2020, background): “With the rapid development in monocular object 6D pose estimation [17, 45], some researchers [8] predict 6D poses of the objects and project predefined grasp poses to the scene.”
- FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation (2021, neutral): “We use the average distance metrics ADD and ADD-S for evaluation.”
- Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation (2019, methodology): “This is substantially higher than PoseCNN[51] which only achieves an mAP of 1.7% without iterative pose refinement (reported in [30]).”
- Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Active Tasks (2018, background): “encompasses the approaches designed to solve various stand-alone vision tasks, e.g. depth estimation [18,43], object classification [42,32], detection [64,25], segmentation [70,30,35], pose estimation [92,9,87], etc. The approaches use various levels of supervision [42,55,14,7], but the common charac”