3D fully convolutional network for vehicle detection in point cloud

By Bo Li. 2016

485 citations recorded by the source corpus.

2D fully convolutional network has been recently successfully applied to the object detection problem on images. In this paper, we extend the fully convolutional network based detection techniques to 3D and apply it to point cloud data. The proposed approach is verified on the task of vehicle detection from lidar point cloud for autonomous driving. Experiments on the KITTI dataset shows significant performance improvement over the previous point cloud based detection approaches.

Subfields: Automotive Engineering, Computational Mechanics, Computer Vision and Pattern Recognition, Environmental Engineering, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Environmental Engineering

In 3D Vision & Pose

Citations in the corpus

VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection (2017, methodology): “Here, we aim to directly estimate the oriented 3D box and normalize Δx and Δy homogeneously with the diagonal d, which is different from [34, 40, 22, 21, 4, 3, 5].”
Multi-view 3D Object Detection Network for Autonomous Driving (2016, methodology): “ect Detection. We ﬁnally evaluate 2D detection performance on KITTI test set. Results are shown in Table5. Among the LIDAR-based methods, our “BV+FV” approach outperforms the recently proposed 3D FCN [16] method by 10.31% AP 2D in the hard setting. In overall, image-based methods usually perform bet”
Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network For Lidar 3D Vehicle Detection (2018, methodology): “Li [4] discretizes point clouds into square grids, and then employs a 3D convolution neural network that outputs an objectness map and a bounding box map for each grid.”
PIXOR: Real-time 3D Object Detection from Point Clouds (2018, methodology): “We compare with 3D object detectors that use LIDAR on KITTI benchmark: VeloFCN [20], 3D FCN [19] and MV3D [3].”
Fast Point R-CNN (2019, background): “Several encoding techniques [32, 33, 17, 18] provide other representations to preserve more information.”
Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal (2019, methodology): “In order to leverage the expertise in 2D object detection, existing methods convert 3D point clouds either into 2D images by view projection [4]–[6], or into regular grids of voxels by quantization [7]–[10].”
Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net (2018, methodology): “Li [15] used 3D point cloud data and proposed to use 3D convolutions on a voxelized representation of point clouds.”
PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation (2017, background): “This includes either projecting the point cloud onto the image [11] or voxelizing the point cloud [32, 17].”
CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection (2020, methodology): “The majority of LiDAR-based methods either use 3D voxels [12, 35] or 2D projections [13, 5, 29, 31] for point cloud representation.”
Frustum PointNets for 3D Object Detection from RGB-D Data (2017, methodology): “Again, our method significantly outperforms previous works which include DoBEM [35] and MV3D [5] that use CNNs on projected LiDAR images, as well as 3D FCN [14] that uses 3D CNNs on voxelized point cloud.”
MVX-Net: Multimodal VoxelNet for 3D Object Detection (2019, methodology): “Some of the techniques ([38], [9], [35], [35], [17]) represent 3D point cloud data using voxel occupancy grid representation, followed by the use of 3D convolutions to compute the 3D bounding boxes.”
Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges (2019, background): “[29], [135]) is usually very time-consuming and infeasible for online autonomous driving.”
HDNET: Exploiting HD Maps for 3D Object Detection (2018, background): “Some detectors [13, 14, 15] search objects in 3D space densely via sliding window.”
Complex-YOLO: Real-time 3D Object Detection on Point Clouds (2018, methodology): “ Chen et al. [5]. The basic idea is the projection of Lidar point clouds into voxel based RGB-maps using handcrafted features, like points density, maximum height and a representative point intensity [9]. To achieve highly accurate results, they use a multi-view approach based on a Lidar birds-eye-v”
YOLO3D: End-to-end real-time 3D Oriented Object Bounding Box Detection from LiDAR Point Cloud (2018, background): “gms dier in data preprocessing. Some approaches project point cloud in 2D view (Bird-View, Front-View) such as[4] and [5]; and some [6] convert the point cloud to a front view depth map. Others like [7] and [8], convert the point cloud to voxels producing a discrete square grid. The Second one is th”
Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Review (2020, methodology): “These frameworks can be applied to several tasks specifically for AVs such as: segmentation and scene understanding [10–12], object detection [13, 14], and classification [10, 15, 16].”
Stereo R-CNN Based 3D Object Detection for Autonomous Driving (2019, methodology): “[6, 26, 15, 20, 31] utilize structured voxel grid representation to quantize the raw point cloud data, then use either 2D or 3D CNN to detect 3D object, while[20] takes multiple frames as input and generates 3D detection, tracking and motion forecasting simultaneously.”
SBNet: Sparse Blocks Network for Fast Inference (2018, unknown): “8 76.01 Dense No Mask 0% 1.0 73.28 SBNet +PSP PSP Mask 86% 2.66 73.01 Table 6: KITTI Bird’s Eye View (BEV) 2017 Benchmark Model Moderate Easy Hard Avg. Time DoBEM [37] 36.95 36.49 38.10 600 ms 3D FCN [23] 62.54 69.94 55.94 > 5 s MV3D [1] 77.00 85.82 68.94 240 ms Dense 77.05 81.70 72.95 47.3 ms SB”
Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving (2019, background): “Most work on 3D object detection operates on 3D point clouds from LiDAR as input (Li, 2017; Li et al., 2016; Meyer et al., 2019b; Yang et al., 2018a; Du et al., 2018; Shi et al., 2019; Engelcke et al., 2017; Yan et al., 2018; Lang et al., 2019).”