Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection

By Benjin Zhu, Zhengkai Jiang, Xiangxin Zhou, Zeming Li, Gang Yu. arXiv.org, 2019

587 citations recorded by the source corpus.

This report presents our method which wins the nuScenes3D Detection Challenge [17] held in Workshop on Autonomous Driving(WAD, CVPR 2019). Generally, we utilize sparse 3D convolution to extract rich semantic features, which are then fed into a class-balanced multi-head network to perform 3D object detection. To handle the severe class imbalance problem inherent in the autonomous driving scenarios, we design a class-balanced sampling and augmentation strategy to generate a more balanced data distribution. Furthermore, we propose a balanced group-ing head to boost the performance for the categories withsimilar shapes. Based on the Challenge results, our methodoutperforms the PointPillars [14] baseline by a large mar-gin across all metrics, achieving state-of-the-art detection performance on the nuScenes dataset. Code will be released at CBGS.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/565166a12e9df19bb4c9a1abd078c6cd6f4bb39f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Tracking Objects as Points (2020, background): “3D trackers replace the object detection component in standard tracking systems with 3D detection from monocular images [30] or 3D point clouds [37, 57].”
3D Object Detection with Pointformer (2020, methodology): “We show that our novel Transformer based model, Pointformer, can be used as a drop-in replacement for voxel-based detector, CBGS [42] and point-based detectors, VoteNet [19] and PointRCNN [25].”
3D Multi-Object Tracking: A Baseline and New Evaluation Metrics (2019, methodology): “Here, we experiment with [2], [28] on KITTI and [29] on nuScenes.”
PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images (2022, methodology): “The DETR3D, BEVDet and PETR are trained with CBGS [45].”
DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries (2021, methodology): “# : trained with CBGS [39] Method The training procedure takes roughly 18 hours.”
nuScenes: A Multimodal Dataset for Autonomous Driving (2019, methodology): “Megvii, PointPillars and MonoDIS achieve an sAMOTA of 17 .”
BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View (2021, methodology): “Following CenterPoint [48], all models are trained with CBGS [54].”
Center-based 3D Object Detection and Tracking (2020, methodology): “We use the same network designs and training schedules as prior works [42, 65].”
Multimodal Virtual Point 3D Detection (2021, methodology): “We also adopt the class-balanced resampling and class-grouped heads in [76] to improve the average density of rare classes.”
PointPainting: Sequential Fusion for 3D Object Detection (2019, background): “However, MEGVII’s network [35] is impractical for a realtime system since it is an extremely large two stage network that requires high resolution inputs and uses multi-scale inputs and ensembles for test evaluation.”
3D-CVF: Generating Joint Camera and LiDAR Features Using Cross-View Spatial Feature Fusion for 3D Object Detection (2020, methodology): “We expand the train data split using DS sampling [31] to handle the class imbalance issue of nuScenes dataset.”
TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers (2022, methodology): “LiDAR-only 3D Detection aims to predict 3D bounding boxes of objects in given point clouds [3, 4, 27, 28, 38, 46, 53, 66, 68, 69].”
Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection (2023, methodology): “For fair comparisons, all models are trained with 24 epochs without CBGS [60] and evaluated on the nuScenes [1] val set.”
BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework (2022, background): “Note that no data augmentation (i.e., ﬂipping, rotation, or CBGS [67]) is applied when multi-view image input is involved.”