CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detection

By Su Pang, Daniel Morris, Hayder M. S. Radha. IEEE/RJS International Conference on Intelligent RObots and Systems, 2020

514 citations recorded by the source corpus.

There have been significant advances in neural networks for both 3D object detection using LiDAR and 2D object detection using video. However, it has been surprisingly difficult to train networks to effectively use both modalities in a way that demonstrates gain over single-modality networks. In this paper, we propose a novel Camera-LiDAR Object Candidates (CLOCs) fusion network. CLOCs fusion provides a low-complexity multi-modal fusion framework that significantly improves the performance of single-modality detectors. CLOCs operates on the combined output candidates before Non-Maximum Suppression (NMS) of any 2D and any 3D detector, and is trained to leverage their geometric and semantic consistencies to produce more accurate final 3D and 2D detection results. Our experimental evaluation on the challenging KITTI object detection benchmark, including 3D and bird's eye view metrics, shows significant improvements, especially at long distance, over the state-of-the-art fusion based methods. At time of submission, CLOCs ranks the highest among all the fusion-based methods in the official KITTI leaderboard. We will release our code upon acceptance.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/20454697ea082975db2503a52418efb8f65b8ae6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

1. Weiliang Tang
2. Chi-Wing Fu
3. Li Jiang
4. Wu Zheng
5. Xin Lai
6. Xirong Li
7. Rui Qian

In Computer Vision and Pattern Recognition

1. Weiliang Tang
2. Zhi Tang
3. Chi-Wing Fu
4. Zhongyu Xia
5. T. Tang
6. Zhiwei Lin
7. Li Jiang
8. Wu Zheng

Citations in the corpus

• BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework (2022, background): “In the early stage of perception systems, people design separate deep models for each sensor [37, 38, 59, 16, 53], and fuse the information via post-processing approaches [32].”
• Multimodal Virtual Point 3D Detection (2021, unknown): “Multi-modal 3D Object Detection fuses information of Lidar and color cameras [20, 28, 28, 29, 35, 37, 53, 67, 71, 72].”
• TransFuser: Imitation With Transformer-Based Sensor Fusion for Autonomous Driving (2022, unknown): “Most sensor fusion works consider perception tasks, e.g. object detection [14], [15], [16], [18], [19], [20], [21], [22], [23], [47], [48], [49], [50], [51], [52], [53], [54], [55], [56], [57], [58], [59], [60] and motion forecasting [24], [25], [26], [27], [28], [29], [30], [49], [61], [62].”
• 3D Object Detection for Autonomous Driving: A Survey (2021, methodology): “[32]—TIP’2019 Arnold et al.[34]—TITS’2019 This paper Guo et al.”
• SE-SSD: Self-Ensembling Single-Stage Object Detector From Point Cloud (2021, methodology): “51 75 CLOCs PVCas [19] IROS 2020 LiDAR+RGB 88.”
• PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection (2021, unknown): “45 - - - - CLOCs [90] IROS 2020 RGB + LiDAR 88.”