BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth

By Mahdi Rad, Vincent Lepetit. IEEE International Conference on Computer Vision, 2017

868 citations recorded by the source corpus.

We introduce a novel method for 3D object detection and pose estimation from color images only. We first use segmentation to detect the objects of interest in 2D even in presence of partial occlusions and cluttered background. By contrast with recent patch-based methods, we rely on a “holistic” approach: We apply to the detected objects a Convolutional Neural Network (CNN) trained to predict their 3D poses in the form of 2D projections of the corners of their 3D bounding boxes. This, however, is not sufficient for handling objects from the recent T-LESS dataset: These objects exhibit an axis of rotational symmetry, and the similarity of two images of such an object under two different poses makes training the CNN challenging. We solve this problem by restricting the range of poses used for training, and by introducing a classifier to identify the range of a pose at run-time before estimating it. We also use an optional additional step that refines the predicted poses. We improve the state-of-the-art on the LINEMOD dataset from 73.7% [2] to 89.3% of correctly registered RGB frames. We are also the first to report results on the Occlusion dataset [1 ] using color images only. We obtain 54% of frames passing the Pose 6D criterion on average on several sequences of the T-LESS dataset, compared to the 67% of the state-of-the-art [10] on the same sequences which uses both color and depth. The full approach is also scalable, as a single network can be trained for multiple objects simultaneously.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/fe25d4c99e39ce28107b56adaa3b5a43219a3a8a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

PointNetLK: Robust & Efficient Point Cloud Registration Using PointNet (2019, background): “On the other hand, in applications such as robotic manipulation, pose is often decoupled into rotation and translation components and each is inferred independently [36, 37, 14, 44, 28, 16].”
H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactions (2019, methodology): “ia, over interaction classes. In the following sections, we describe each of these components in more detail. 3.2. Joint 3D Hand-Object Pose Estimation In the context of rigid object pose estimation, [46,66] regress the 2D locations of 8 keypoints – namely, the projections of the 8 corners of the ob”
HybridPose: 6D Object Pose Estimation Under Hybrid Representations (2020, background): “[38], BB8 [36], Pix2Pose [30], PVNet [34], CDPN [20], and DPOD [44].”
CosyPose: Consistent multi-view multi-object 6D pose estimation (2020, methodology): “Object pose estimation is one of the oldest computer vision problems [1–3], yet it remains an active area of research [4–11].”
CDPN: Coordinates-Based Disentangled Pose Network for Real-Time RGB-Based 6-DoF Object Pose Estimation (2019, methodology): “proaches [20, 8, 3] refined with depth and ICP.”
GDRNPP: A Geometry-Guided and Fully Learning-Based Object Pose Estimator (2021, neutral): “For instance, BB8 [37] and YOLO6D [38] compute the 2D projections of a set of ﬁxed control points (e.g.the 3D corners of the encapsulating bounding box).”
ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shape (2018, methodology): “[34] empirically show the superiority of this proxy loss over standard regression of the 6 degrees of freedom.”
DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion (2019, methodology): “d objects in 13 videos. It is widely adopted by both classical methods [4,8,36] and recent learning-based approaches [17,30,33]. We use the same training and testing set as prior learning-based works [17,24,33] without additional synthetic data and compare with the best ICP-reﬁned results of the sta”
DeepIM: Deep Iterative Matching for 6D Pose Estimation (2018, methodology): “The state-of-the-art methods (Rad and Lepetit 2017; Kehl et al. 2017; Tekin et al. 2017; Xiang et al. 2018; Tremblay et al. 2018) augment deep learning based object detection or segmentation methods (Girshick 2015; Long et al. 2015; Liu et al. 2016; Redmon et al. 2016) for 6D pose estimation.”
Leveraging Photometric Consistency Over Time for Sparsely Supervised Hand-Object Reconstruction (2020, neutral): “For object pose estimation from RGB images, the recent trend is to use convolutional neural networks (CNNs) to predict the 2D locations of the object’s 3D bounding box in image space [20, 33, 44].”
BOP: Benchmark for 6D Object Pose Estimation (2018, background): “[34,24,18,2,36,21,27,25], but it is unclear which methods perform well and in which scenarios.”
On Pre-Trained Image Features and Synthetic Images for Deep Learning (2017, methodology): “bjects in real images. The top-left image shows the CAD model we used to learn to detect the object in the three other images. Several approaches have tried to overcome this domain gap. For instance, [5,7,17] use synthetic images in addition to real ones to boost performance. While this usually resu”
Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review (2020, methodology): “There exists another kind of methods (Rad and Lepetit 2017; Tekin et al.”
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes (2017, methodology): “by rendering the corresponding 3D model. Recently, 2D object detection methods are used as template matching and augmented for 6D pose estimation, especially with deep learning-based object detectors [28, 23, 16, 29]. Templatebased methods are useful in detecting texture-less objects. However, they ”
DPOD: 6D Pose Object Detector and Refiner (2019, methodology): “BB8 [26] uses a three stage approach.”
DexMV: Imitation Learning for Dexterous Manipulation from Human Videos (2021, background): “One line of work focus on object pose estimation [36, 56, 85, 79, 50, 30, 27] and hand pose estimation [87, 31, 76, 23, 5, 11, 26, 38, 42].”
PVNet: Pixel-Wise Voting Network for 6DoF Pose Estimation (2018, methodology): “Some recent methods [29, 30, 36] use CNNs to first regress 2D keypoints and then compute 6D pose parameters using the Perspective-n-Point (PnP) algorithm.”
Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation (2019, methodology): “Furthermore, symmetric objects are usually handled by limiting the range of viewpoints, which sometimes requires additional treatments, e.g., training a CNN for classifying view ranges [24].”
Imbalance Problems in Object Detection: A Review (2019, background): “surveillance [1], [2], autonomous driving [3], [4], medical decision making [5], [6], and many problems in robotics [7], [8], [9], [10], [11], [12].”