Data-driven 3D Voxel Patterns for object category recognition

By Yu Xiang, Wongun Choi, Yuanqing Lin, Silvio Savarese. 2015

317 citations recorded by the source corpus.

Despite the great progress achieved in recognizing objects as 2D bounding boxes in images, it is still very challenging to detect occluded objects and estimate the 3D properties of multiple objects from a single image. In this paper, we propose a novel object representation, 3D Voxel Pattern (3DVP), that jointly encodes the key properties of objects including appearance, 3D shape, viewpoint, occlusion and truncation. We discover 3DVPs in a data-driven way, and train a bank of specialized detectors for a dictionary of 3DVPs. The 3DVP detectors are capable of detecting objects with specific visibility patterns and transferring the meta-data from the 3DVPs to the detected objects, such as 2D segmentation mask, 3D pose as well as occlusion or truncation boundaries. The transferred meta-data allows us to infer the occlusion relationship among objects, which in turn provides improved object recognition results. Experiments are conducted on the KITTI detection benchmark [17] and the outdoor-scene dataset [41]. We improve state-of-the-art results on car detection and pose estimation with notable margins (6% in difficult data of KITTI). We also verify the ability of our method in accurately segmenting objects from the background and localizing them in 3D.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

Citations in the corpus

VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection (2017, background): “Given that images provide detailed texture information, many algorithms infered the 3D bounding boxes from 2D images [4, 3, 44, 45, 46, 38].”
Multi-view 3D Object Detection Network for Autonomous Driving (2016, methodology): “3DVP [24] introduces 3D voxel patterns and employ a set of ACF detectors to do 2D detection and 3D pose estimation.”
A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection (2016, neutral): “[35] 10s 68.02 56.48 44.18 47.74 39.36 35.95 35.04 27.50 26.21 DPM-VOC-VP [36] 8s 74.95 64.71 48.76 59.48 44.86 40.37 42.43 31.08 28.23 SubCat [16] 0.7s 84.14 75.46 59.71 54.67 42.34 37.95 - - - 3DVP [37] 40s 87.46 75.77 65.38 - - - - - - AOG [38] 3s 84.80 75.94 60.70 - - - - - - Faster-RCNN [9] 2s ”
Monocular Object Instance Segmentation and Depth Ordering with CNNs (2015, methodology): “In [38], 2D boxes are detected and the method reasons about learned 3D occlusion patterns within each box.”
DenseBox: Unifying Landmark Localization with End to End Object Detection (2015, neutral): “eo information for training and testing. Recently a method named “DJML overtakes all other methods. 9 Method Moderate Easy Hard Regionlets [23] 76.45% 84.75% 59.70% AOG [19] 74.26% 84.24% 60.51% 3DVP [40] 75.77% 87.46% 65.38% spCov LBP 77.40% 87.19% 60.60% DeepInsight 84.40% 84.59% 76.09% NIPS ID 33”
Fine-Grained Categorization and Dataset Bootstrapping Using Deep Metric Learning with Humans in the Loop (2015, methodology): “1) Rather than using softmax loss, we aim to find a low-dimensional feature embedding for classification.”
Learning Efficient Object Detection Models with Knowledge Distillation (2017, background): “Among them, KITTI and PASCAL are relatively small datasets that contain less object [39] and [24] for analysis.”
BirdNet: A 3D Object Detection Framework from LiDAR Information (2018, methodology): “Similarly, 3D voxel patterns have been used to estimate position and orientation of objects in the scene [8].”
Fusing Bird View LIDAR Point Cloud and Front View Camera Image for Deep Object Detection (2017, methodology): “By adding the prior part and dimension knowledge of cars and cyclists, the earlier works such as 3DVP [22] and Mono3D produces 3D bounding boxes.”
ObjectNet3D: A Large Scale Database for 3D Object Recognition (2016, background): “[9,24,42,33,15,39]), as the field has benefited from the introduction of several important databases that provide 3D annotations to 2D objects.”
Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art (2017, methodology): “In contrast to the Bayesian and min-cost formulation, Xiang et al. (2015a) consider the tracking problem as a Markov decision process (MDP).”
Disentangling Monocular 3D Object Detection (2019, background): “pectively, of many recent monocular 3D detection approaches. When evaluating on the validation set, we consider the split deﬁned in [5], as is done in all the baselines. Please note that the works in [40,24,41] are using yet another training/validation split, rendering their results incomparable to ”
3D-RCNN: Instance-Level 3D Object Reconstruction via Render-and-Compare (2018, methodology): “When it comes to shape, most methods either estimate only 3D bounding boxes [36, 6, 12], or coarse wire-frame skeletons [29, 54, 63, 62], or represent shape via an exemplar mesh chosen from a small set of meshes [4, 56, 35, 2].”
Delving into Localization Errors for Monocular 3D Object Detection (2021, methodology): “Specifically, [38, 26] use the CAD models as shape templates to get better object geometry.”
A General Pipeline for 3D Detection of Vehicles (2018, methodology): “[9], [10], [11] and [12], defined 3D car models with occlusion patterns, carried out detection of the patterns in the 2D image and recovered the 3D occluded structure by reasoning through a MAP (maximum a posteriori) framework.”
RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving (2020, unknown): “…Depth Shape/CAD Segmentation Mono3D [7] 3DOP [8], stereoRCNN [22] MF3D [42], Peseudo-LiDAR [39], MonoPSR [17] AM3D[27] Mono3D++ [14],Deep-MENTA [6], 3DVP [40] Deep3DBox [28],GS3D [20],MonoGRNet [33], FQNet[25], M3D-RPN[4] Shift-RCNN [30] Ours(RTM3D) Image-only in Monocular 3D Object Detection.”
3D Bounding Box Estimation Using Deep Learning and Geometry (2016, methodology): “Our method is the only one that does not rely on computing additional features such as stereo, semantic segmentation, instance segmentation and does not need preprocessing as in [24] and [23].”
Exploit All the Layers: Fast and Accurate CNN Object Detector with Scale Dependent Pooling and Cascaded Rejection Classifiers (2016, methodology): “Since the ground-truth annotation of testing set is not publicly available, we use the training/validation split of [38] for the analysis.”