Focal Sparse Convolutional Networks for 3D Object Detection

By Yukang Chen, Yanwei Li, X. Zhang, Jian‐Yuan Sun, Jiaya Jia. Computer Vision and Pattern Recognition, 2022

313 citations recorded by the source corpus.

Non-uniformed 3D sparse data, e.g., point clouds or voxels in different spatial positions, make contribution to the task of 3D object detection in different ways. Existing basic components in sparse convolutional networks (Sparse CNNs) process all sparse data, regardless of regular or submanifold sparse convolution. In this paper, we introduce two new modules to enhance the capability of Sparse CNNs, both are based on making feature sparsity learnable with position-wise importance prediction. They are focal sparse convolution (Focals Conv) and its multi-modal variant of focal sparse convolution with fusion, or Focals Conv-F for short. The new modules can readily substitute their plain counterparts in existing Sparse CNNs and be jointly trained in an end-to-end fashion. For the first time, we show that spatially learnable sparsity in sparse convolution is essential for sophisticated 3D object detection. Extensive experiments on the KITTI, nuScenes and Waymo benchmarks validate the effectiveness of our approach. Without bells and whistles, our results outperform all existing single-model entries on the nuScenes test benchmark. Code and models are at github.com/dvlab-research/FocalsConv.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/fdd82bf3177aa309bc2c79cac8482d8f3c4e8190

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation (2022, methodology): “Among all these methods, PointPainting [54], PointAugmenting [55], MVP [68], FusionPainting [64], AutoAlign [12] and FocalSparseCNN [9] are (LiDAR) input-level decoration, while Deep Continuous Fusion [27] and DeepFusion [23] are feature-level decoration.”
BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework (2022, background): “1, these methods can be divided into two categories depending on their fusion mechanism, (a) point-level fusion where one queries the image features via the raw LiDAR points and then concatenates them back as additional point features [17, 43, 47, 60]; (b) feature-level fusion where one firstly proj”
3D Object Detection for Autonomous Driving: A Comprehensive Survey (2022, background): “There is also a series of works trying to improve the sparse operators [49], extend [333] into a two-stage detector [254, 57], and introduce the Transformer [280] architecture into voxel-based detection [187, 70].”
Unifying Voxel-based Representation with Transformer for 3D Object Detection (2022, methodology): “For voxel-based methods, point clouds are transformed into regular grids and processed with 3D sparse convolutions [17, 18] or 2D convolutions [19, 20, 21] directly.”