SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again

By Wadim Kehl, Fabian Manhardt, Federico Tombari, Slobodan Ilic, Nassir Navab. IEEE International Conference on Computer Vision, 2017

1,003 citations recorded by the source corpus.

We present a novel method for detecting 3D model instances and estimating their 6D poses from RGB data in a single shot. To this end, we extend the popular SSD paradigm to cover the full 6D pose space and train on synthetic model data only. Our approach competes or surpasses current state-of-the-art methods that leverage RGBD data on multiple challenging datasets. Furthermore, our method produces these results at around 10Hz, which is many times faster than the related methods. For the sake of reproducibility, we make our trained networks and detection code publicly available.

Subfields: Computer Vision and Pattern Recognition, Industrial and Manufacturing Engineering

View Paper: https://www.semanticscholar.org/paper/18689c0acb559776b1ef2ea3e8684f69358fec60

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Industrial and Manufacturing Engineering

Citations in the corpus

PointNetLK: Robust & Efficient Point Cloud Registration Using PointNet (2019, background): “On the other hand, in applications such as robotic manipulation, pose is often decoupled into rotation and translation components and each is inferred independently [37, 38, 15, 45, 29, 17].”
GraspNet-1Billion: A Large-Scale Benchmark for General Object Grasping (2020, methodology): “Rotation and Width It has been proved in previous literature [17] that for predicting in-plane rotation, classification could achieve better results than regression.”
Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review (2020, methodology): “42 SSD-6D [Kehl et al., 2017]+Refine 76.”
PVN3D: A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation (2019, methodology): “On the ADD(S) metric, our model outperforms PoseCNN+ICP [52] by 6.4% RGB RGBD PoseCNNDeepIM [26,52] PVNet[37] CDPN[27] ImplicitICP[45] SSD-6DICP[22] Point-Fusion[50] DF(per-pixel)[50] DF(iterative)[50 and exceeds DF(iterative) [50] by 5.7%.”
Deep Template-based Object Instance Detection (2021, background): “proaches such as [4, 22, 44, 18] are all learning-based methods that were specifically trained on the objects.”
DPOD: 6D Pose Object Detector and Refiner (2019, methodology): “We evaluate the quality of 6DoF pose estimation following the procedure suggested at SSD6D [15] also used in other papers.”
Shape Prior Deformation for Categorical 6D Object Pose and Size Estimation (2020, background): “[10,37] extend 2D object detection network such that it can predict orientation as an add-on to the identity and 2D bounding box of the object.”
PETR: Position Embedding Transformation for Multi-View 3D Object Detection (2022, background): “Many previous works [6,33,20,21,41,19,2,49,48] perform 3D object detection in the image view.”
Self-supervised 6D Object Pose Estimation for Robot Manipulation (2019, background): “Deep neural networks further boost the performance by learning more discriminative features and classifiers [1], [2], [8], [9], [19].”
Real-Time Seamless Single Shot 6D Object Pose Prediction (2017, methodology): “Deep learning techniques have recently been used to address these limitations [10, 25].”
Few-Shot Object Detection and Viewpoint Estimation for Objects in the Wild (2020, methodology): “Deep-learning methods for viewpoint estimation follow roughly three different paths: direct estimation of Euler angles [55, 50, 33, 24, 62, 63], template-based matching [20, 32, 51], and keypoint detection relying on 3D bounding box corners [39, 52, 13, 34, 36] or semantic keypoints [35, 65].”
PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images (2022, background): “BEV segmentation focus on the perception in the BEV view.”
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes (2017, methodology): “Recently, 2D object detection methods are used as template matching and augmented for 6D pose estimation, especially with deep learning-based object detectors [28, 23, 16, 29].”
DexMV: Imitation Learning for Dexterous Manipulation from Human Videos (2021, background): “One line of work focus on object pose estimation [36, 56, 85, 79, 50, 30, 27] and hand pose estimation [87, 31, 76, 23, 5, 11, 26, 38, 42].”
DeceptionNet: Network-Driven Domain Randomization (2019, methodology): “[21, 29] tackle this problem by rendering objects on top of images from large-scale datasets (e.”
DeepIM: Deep Iterative Matching for 6D Pose Estimation (2018, methodology): “The state-of-the-art methods (Rad and Lepetit 2017; Kehl et al. 2017; Tekin et al. 2017; Xiang et al. 2018; Tremblay et al. 2018) augment deep learning based object detection or segmentation methods (Girshick 2015; Long et al. 2015; Liu et al. 2016; Redmon et al. 2016) for 6D pose estimation.”
Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D (2020, methodology): “A standard technique is to apply a mature 2D object detector in the image plane and then train a second network to regress 2D boxes into 3D boxes [12,26,31,27].”
ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shape (2018, methodology): “We chose to encode the rotation as a 4D quaternion and the translation as the projective 2D object centroid (similar to [31, 20, 40]) together with the associated depth.”
CosyPose: Consistent multi-view multi-object 6D pose estimation (2020, methodology): “A convolutional neural network (CNN) can be used to detect object features in 2D [4,6,18,21,22] or to directly find 2D-to-3D correspondences [5,7,8,23].”
Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation (2019, methodology): “In addition, the public code and trained weights of SSD-6D [14] are used to derive 2D detection results while ignoring pose predictions of the network.”