3D Object Proposals Using Stereo Imagery for Accurate Object Class Detection

By Xiaozhi Chen, Kaustav Kundu, Yukun Zhu, Huimin Ma, Sanja Fidler, Raquel Urtasun. 2016

404 citations recorded by the source corpus.

The goal of this paper is to perform 3D object detection in the context of autonomous driving. Our method aims at generating a set of high-quality 3D object proposals by exploiting stereo imagery. We formulate the problem as minimizing an energy function that encodes object size priors, placement of objects on the ground plane as well as several depth informed features that reason about free space, point cloud densities and distance to the ground. We then exploit a CNN on top of these proposals to perform object detection. In particular, we employ a convolutional neural net (CNN) that exploits context and depth information to jointly regress to 3D bounding box coordinates and object pose. Our experiments show significant performance gains over existing RGB and RGB-D object proposal methods on the challenging KITTI benchmark. When combined with the CNN, our approach outperforms all existing results in object detection and orientation estimation tasks for all three KITTI object classes. Furthermore, we experiment also with the setting where LIDAR information is available, and show that using both LIDAR and stereo leads to the best result.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Industrial and Manufacturing Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Industrial and Manufacturing Engineering

Citations in the corpus

Multi-view 3D Object Detection Network for Autonomous Driving (2016, methodology): “For Mono3D and 3DOP, we use 3D box regression [5] instead of 2D box regression used in the original implementation.”
PIXOR: Real-time 3D Object Detection from Point Clouds (2018, methodology): “Camera based approaches utilize either monocular [1] or stereo images [2].”
Seeing Through Fog Without Seeing Fog: Deep Multimodal Sensor Fusion in Unseen Adverse Weather (2019, methodology): “.3 shows estimation results on synthetic KTTI data, recall Sec.5.2. For these evaluations, we took models pretrained on ImageNet and ﬁne-tuned for fog density estimation with an identical split as in [6] for all images where a history of at least 10 framews was available. Using temporal image sequen”
GS3D: An Efficient 3D Object Detection Framework for Autonomous Driving (2019, methodology): “[3], can obtain a depth map using the disparity computed from the images of different views.”
DynaSLAM II: Tightly-Coupled Multi-Object Tracking and SLAM (2020, background): “The CNNs of Chen et al. [28] and specially of Li et al. [10] achieve excellent results thanks to the single-view network accuracy itself and the multi-view reﬁnement approach of the latter one, to the detriment of a generality loss.”
Learning 3D Shape Completion from Laser Scan Data with Weak Supervision (2018, methodology): “We used KITTI’s Velodyne point clouds from the 3D object detection benchmark and the training/validation split of [6] (7140/7118 samples).”
Learning Depth-Guided Convolutions for Monocular 3D Object Detection (2019, background): “Deep-MANTA [2] X X 3DVP [53] X X Mono3D [4, 6] X X X X Mono3D++ [17] X X X Table 6.”
Pseudo-LiDAR From Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving (2018, methodology): “We modify the algorithm, following [6, 30], to generate five frontal-view feature maps, including 3D pixel locations, disparity, and Euclidean distance to the camera.”
HDNET: Exploiting HD Maps for 3D Object Detection (2018, methodology): “For the ground plane counterpart, we use the publicly available ground plane results from 3DOP [21], which were generated by road segmentation and RANSAC fitting.”
Deep Continuous Fusion for Multi-sensor 3D Object Detection (2018, background): “Images, on the other hand, provide dense measurements, but precise 3D localization is hard due to the loss of depth information caused by perspective projection, particularly when using monocular cameras [3,5].”
Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net (2018, methodology): “[2] used stereo images to perform 3D detection.”
MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationships (2020, methodology): “Most existing methods on 3D object detection require accurate depth information, which can be obtained from either 3D LiDARs [8, 30, 34, 35, 23, 45] or multicamera systems [6, 7, 20, 29, 32, 41].”
Multi-Task Multi-Sensor Fusion for 3D Object Detection (2019, background): “Over the past few years we have seen a plethora of methods that tackle the problem of 3D object detection from monocular images [2, 31], stereo cameras [4] or LiDAR point clouds [36, 34, 16].”
Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges (2019, background): “Deep learning approaches have set the benchmark on many popular object detection datasets, such as PASCAL VOC [17] and COCO [18], and have been widely applied in autonomous driving, including detecting traffic lights [19]–[22], road signs [23]–[25], people [26]–[28],”