StarNet: Targeted Computation for Object Detection in Point Clouds

By Jiquan Ngiam, Benjamin Caine, Wei Han, Brandon Yang, Yuning Chai, Pei Sun, Yin Zhou, Xinyu Yi, Ouais Alsharif, Patrick Nguyen and 3 more. 2019

130 citations recorded by the source corpus.

Detecting objects from LiDAR point clouds is an important component of self-driving car technology as LiDAR provides high resolution spatial information. Previous work on point-cloud 3D object detection has re-purposed convolutional approaches from traditional camera imagery. In this work, we present an object detection system called StarNet designed specifically to take advantage of the sparse and 3D nature of point cloud data. StarNet is entirely point-based, uses no global information, has data dependent anchors, and uses sampling instead of learned region proposals. We demonstrate how this design leads to competitive or superior performance on the large Waymo Open Dataset and the KITTI detection dataset, as compared to convolutional baselines. In particular, we show how our detector can outperform a competitive baseline on Pedestrian detection on the Waymo Open Dataset by more than 7 absolute mAP while being more computationally efficient. We show how our redesign---namely using only local information and using sampling instead of learned proposals---leads to a significantly more flexible and adaptable system: we demonstrate how we can vary the computational cost of a single trained StarNet without retraining, and how we can target proposals towards areas of interest with priors and heuristics. Finally, we show how our design allows for incorporating temporal context by using detections from previous frames to target computation of the detector, which leads to further improvements in performance without additional computational cost.

Subfields: Automotive Engineering, Computer Vision and Pattern Recognition, Environmental Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Computer Vision and Pattern Recognition

In Environmental Engineering

Citations in the corpus

Randaugment: Practical automated data augmentation with a reduced search space (2019, methodology): “Future work will study how this method applies to other machine learning domains, where data augmentation is known to improve predictive performance, such as image segmentation [2], 3-D perception [29], speech recognition [14] or audio recognition [13].”
nuScenes: A Multimodal Dataset for Autonomous Driving (2019, background): “A similar conclusion was reached by [56, 59] with [59] suggesting that the KITTI leaderboard reflects the data aug.”
Center-based 3D Object Detection and Tracking (2020, unknown): “PV-RCNN [44] proposes the Voxel-Set Abstraction (VSA) module, which extends Point-7 56.4 PointPillars [28] 56.6 59.3 PPBA [36] 62.4 66.0 MVF [65] 62.9 65.3 Huang et al. [24] 63.6 AFDet [14] 63.7 CVCNet [7] 65.2 Pillar-OD [52] 69.8 72.5 PV-RCNN [44] 74.”
DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection (2022, background): “Following them, [22, 24, 31, 42] also learn features with PointNet-like [25] layers.”
PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection (2019, methodology): “4.2) and the newly introduced large-scale Waymo Open Dataset [31, 22, 44] (Sec.”
PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection (2021, methodology): “We follow StarNet [88] and MVF [60] to evaluate the models on the LEVEL 1 difficulty for comparing with previous methods.”
Tradeoffs in Data Augmentation: An Empirical Study (2021, unknown): “, 2016) or cut-and-paste new objects (Dwibedi et al., 2017; Fang et al., 2019; Ngiam et al., 2019).”
Affinity and Diversity: Quantifying Mechanisms of Data Augmentation (2020, methodology): “For object detection and image segmentation, one can also use object-centric cropping (Liu et al., 2016) or cut-and-paste new objects (Dwibedi et al., 2017; Fang et al., 2019; Ngiam et al., 2019).”
Improving 3D Object Detection with Channel-wise Transformer (2021, methodology): “Waymo dataset consists of 798 training sequences with around 158,361 LiDAR samples, and 202 validation sequences with 40,077 LiDAR samples.”
Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges (2019, background): “In fact, a recent work [163] states that”
RandAugment: Practical data augmentation with no separate search (2019, methodology): “Future work will study how this method applies to other machine learning domains, where data augmentation is known to improve predictive performance, such as image segmentation [3], 3-D perception [34], speech recognition [18] or audio recognition [17].”