HDNET: Exploiting HD Maps for 3D Object Detection

By Binh Yang, Ming Liang, Raquel Urtasun. Conference on Robot Learning, 2018

347 citations recorded by the source corpus.

In this paper we show that High-Definition (HD) maps provide strong priors that can boost the performance and robustness of modern 3D object detectors. Towards this goal, we design a single stage detector that extracts geometric and semantic features from the HD maps. As maps might not be available everywhere, we also propose a map prediction module that estimates the map on the fly from raw LiDAR data. We conduct extensive experiments on KITTI as well as a large-scale 3D detection benchmark containing 1 million frames, and show that the proposed map-aware detector consistently outperforms the state-of-the-art in both mapped and un-mapped scenarios. Importantly the whole framework runs at 20 frames per second.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/bcfe57e2d05648c7ace15f3e96bb899b3fa262f2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Guided Curriculum Model Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation (2019, methodology): “literature [53], [54]. Maps have also been enriched to be leveraged for other vision tasks beyond localization such as road surface detection [55], [56], navigation [57], [58], object detection [59], [60], tracking [61] and forecasting [62]. This work uses a new form of map-based JOURNAL OF LATEX CL”
End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Clouds (2019, methodology): “HDNET [28] encodes elevation map information together with BEV feature map.”
3D Multi-Object Tracking: A Baseline and New Evaluation Metrics (2019, methodology): “ounding box matters. Prior works mainly focus on processing the LiDAR point cloud inputs. [1,14] divide the point cloud into equally-spaced 3D voxels and apply 3D CNNs for 3D bounding box prediction. [15] converts the point cloud to a bird’s eye view representation for efﬁciency and exploit 2D convo”
LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Driving (2019, methodology): “Most of the state-of-the-art 2D [24, 19] and 3D [5, 15, 27, 17] object detection methods produce a single box with a probability score for each detection.”
PointPillars: Fast Encoders for Object Detection From Point Clouds (2018, methodology): “Following the tremendous advances in deep learning methods for computer vision, a large body of literature has investigated to what extent this technology could be applied towards object detection from lidar point clouds [33, 31, 32, 11, 2, 21, 15, 30, 26, 25].”
Improving 3D Object Detection with Channel-wise Transformer (2021, background): “Another category of methods [3, 13, 34, 35, 28, 15, 12, 16, 17] aimed to voxelize the unstructured point clouds as a regular 2D/3D grid over which conventional CNNs can be easily applied.”
TANet: Robust 3D Object Detection from Point Clouds with Triple Attention (2019, background): “…two categories: the front view based approaches (Song and Chandraker 2015; Chen et al. 2016; Mousavian et al. 2017), and the bird’s eye view based approaches (Chen et al. 2017; Ku et al. 2018; Yang, Luo, and Urtasun 2018; Simon et al. 2018; Li, Zhang, and Xia 2016; Yang, Liang, and Urtasun 2018).”
Predicting Semantic Map Representations From Images Using Pyramid Occupancy Networks (2020, neutral): “In 3D object detection, [27] use ground height prior information from maps to improve the quality of input LiDAR point clouds.”
Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges (2019, neutral): “[121] and Cascas et al.”
Argoverse: 3D Tracking and Forecasting With Rich Maps (2019, background): “While high definition (HD) maps are widely used by motion planning systems, few works explore the use of this strong prior in perception systems [38] despite the fact that the three winning entries of the 2007 DARPA Urban Challenge relied on a DARPA-supplied map – the Route Network Definition File (”
VectorMapNet: End-to-end Vectorized HD Map Learning (2022, neutral): “Recently, semantic map learning is formulated as a semantic segmentation problem (Mattyus et al., 2015) and is solved by using aerial images (M ´ attyus et al., 2016), LiDAR points (Yang et al., 2018), and HD panorama (Wang et al., 2016).”
LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World (2020, methodology): “Specifically, we simulate 100k frames of LiDAR training data using either LiDARsim or CARLA, train a BEV detector [44], and evaluate over KITTI validation set.”
Unifying Voxel-based Representation with Transformer for 3D Object Detection (2022, methodology): “For voxel-based methods, point clouds are transformed into regular grids and processed with 3D sparse convolutions [17, 18] or 2D convolutions [19, 20, 21] directly.”
Multi-Task Multi-Sensor Fusion for 3D Object Detection (2019, methodology): “In BEV detection, we outperform the best detector HDNET [33], which also exploits ground estimation, by 0.”
Structure Aware Single-Stage 3D Object Detection From Point Cloud (2020, background): “[26] proposed to estimate the ground-level to enhance the detector’s awareness of road geometry.”
Learning Lane Graph Representations for Motion Forecasting (2020, methodology): “admap, trac lights and speed limits in rasterized bird’s eye view images. [23] encodes the history of static entities, dynamic entities and semantic map information in a top-down spatial grid. HDNet [38] exploits the road mask as input feature to improve object detection performance. Rasterized maps”
Deep Learning for 3D Point Clouds: A Survey (2019, methodology): “[152] exploited the geometric and semantic prior information provided by High-”
MVX-Net: Multimodal VoxelNet for 3D Object Detection (2019, neutral): “LD. (S: SINGLE MODALITY, M:MULTIMODAL) Method AP BEV (IoU=0.7) AP 3D (IoU=0.7) Easy Med Hard Easy Med Hard MV3D [5] (S) 85.8 77.0 68.9 66.8 52.7 51.3 PIXOR [42] (S) 81.7 77.1 73.0 N/A N/A N/A PIXOR++ [41] (M) 89.4 83.7 78.0 N/A N/A N/A VoxelNet [43] (S) 89.4 79.3 77.4 77.5 65.1 57.7 MV3D [5] (M) 86.”