Vote3Deep: Fast object detection in 3D point clouds using efficient convolutional neural networks

By Martin Engelcke, Dushyant Rao, Dominic Zeng Wang, Chi Hay Tong, Ingmar Posner. 2016

541 citations recorded by the source corpus.

This paper proposes a computationally efficient approach to detecting objects natively in 3D point clouds using convolutional neural networks (CNNs). In particular, this is achieved by leveraging a feature-centric voting scheme to implement novel convolutional layers which explicitly exploit the sparsity encountered in the input. To this end, we examine the trade-off between accuracy and speed for different architectures and additionally propose to use an L1 penalty on the filter activations to further encourage sparsity in the intermediate representations. To the best of our knowledge, this is the first work to propose sparse convolutional layers and L1 regularisation for efficient large-scale processing of 3D data. We demonstrate the efficacy of our approach on the KITTI object detection benchmark and show that VoteSDeep models with as few as three layers outperform the previous state of the art in both laser and laser-vision based approaches by margins of up to 40% while remaining highly competitive in terms of processing time.

Subfields: Computer Vision and Pattern Recognition, Environmental Engineering, Instrumentation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Environmental Engineering

In Instrumentation

Citations in the corpus

VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection (2017, background): “[43, 9] encode each nonempty voxel with 6 statistical quantities that are derived from all the points contained within the voxel.”
OctNet: Learning Deep 3D Representations at High Resolutions (2016, background): “[10] proposed to calculate convolutions at sparse input locations by pushing values to their target locations.”
Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network For Lidar 3D Vehicle Detection (2018, background): “Engelcke et al. [7] use a voting scheme to search possible object locations, and introduce special convolutional layers to tackle with the sparsity of the point clouds.”
PIXOR: Real-time 3D Object Detection from Point Clouds (2018, methodology): “As a result, typical systems [6, 37, 20] only run at 1-2 FPS.”
A Survey on 3D Object Detection Methods for Autonomous Driving Applications (2019, methodology): “Aiming at a more efficient implementation, [55] fixes BB sizes for each class but detects cars, pedestrians and cyclists.”
BirdNet: A 3D Object Detection Framework from LiDAR Information (2018, methodology): “On the one hand, some approaches work with spatial information by turning the 3D space into a voxel grid and applying 3D convolutions [11] [12] [13].”
Joint 3D Proposal Generation and Object Detection from View Aggregation (2017, methodology): “Methods using LIDAR point clouds [7, 19, 18, 8], stereo depth Figure 1: Parameters employed by metrics commonly used to characterize the performance of 3D object detectors.”
PointCLIP: Point Cloud Understanding by CLIP (2021, unknown): “classification [12,17,22,28,37,41], object detection [1,4,13, 29,47,67], semantic segmentation [3,25,35,36,64,68], point cloud recognition and part segmentation [19,42,44,45,56].”
Noise-Resistant Deep Learning for Object Classification in Three-Dimensional Point Clouds Using a Point Pair Descriptor (2018, background): “clouds into 3D voxel grids ([11] and [12], to name a few).”
A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving (2020, unknown): “Those ROI can be extracted by non-deep learning approaches such as clustering [88] and voting [89], or by a neural network such as a Region Proposal Network (RPN).”
Complex-YOLO: An Euler-Region-Proposal for Real-Time 3D Object Detection on Point Clouds (2018, methodology): “Translation of Point-Clouds into voxels or image stacks by using Convolutional Neural Networks (CNN) [2] [3] [4] [6] [8] [9] [25] [26] 3.”
End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Clouds (2019, methodology): “The transformation process is usually hand-crafted, some representative works include Vote3D [17], Vote3Deep [18], 3DFCN [19], AVOD [20], PIXOR [1] and Complex YOLO [21].”
SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filters (2018, methodology): “Vote3Deep [6] uses a feature-centric voting scheme so that the computational cost is proportional to the number of points with non-zero features.”
HDNET: Exploiting HD Maps for 3D Object Detection (2018, background): “Some detectors [13, 14, 15] search objects in 3D space densely via sliding window.”
Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attacks (2021, background): “Occupancy [15], [16], [30], [34], [49], [73]”
Fusing Bird View LIDAR Point Cloud and Front View Camera Image for Deep Object Detection (2017, methodology): “The DNN-based object detection with either LIDAR [2, 3] or camera [4, 5, 6] data as the input has been widely explored by researchers and pushed to a very high single frame accuracy.”