Fusing Bird View LIDAR Point Cloud and Front View Camera Image for Deep Object Detection

By Zining Wang, Wei Zhan, Masayoshi Tomizuka. 2017

29 citations recorded by the source corpus.

We propose a new method for fusing a LIDAR point cloud and camera-captured images in the deep convolutional neural network (CNN). The proposed method constructs a new layer called non-homogeneous pooling layer to transform features between bird view map and front view map. The sparse LIDAR point cloud is used to construct the mapping between the two maps. The pooling layer allows efficient fusion of the bird view and front view features at any stage of the network. This is favorable for the 3D-object detection using camera-LIDAR fusion in autonomous driving scenarios. A corresponding deep CNN is designed and tested on the KITTI bird view object detection dataset, which produces 3D bounding boxes from the bird view map. The fusion method shows particular benefit for detection of pedestrians in the bird view compared to other fusion-based object detection networks.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

1. Andrew Markham
2. Christian Haase-Schuetz
3. Sen Wang
4. R. Clark
5. Werner Wiesbeck
6. Di Feng
7. Klaus C. J. Dietmayer
8. Lars Rosenbaum

In Computer Vision and Pattern Recognition

1. Andrew Markham
2. Agathoniki Trigoni
3. Di Feng
4. Klaus C. J. Dietmayer
5. Christian Haase-Schuetz
6. Sen Wang
7. R. Clark
8. Werner Wiesbeck

Citations in the corpus

• Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network For Lidar 3D Vehicle Detection (2018, background): “Therefore, it is natural to combine these two sensors [10]–[14].”
• Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges (2019, background): “neural networks for car detection; [88], [95], [96], [99]–[102] focus on detecting non-motorized road users (pedestrians or cyclists); [53], [78], [87], [89]–[92], [98] detect both.”
• Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds (2019, methodology): “Detection performance is further improved by fusing RGB images in [3; 54; 36; 52].”