Improving 3D Object Detection with Channel-wise Transformer

By Hualian Sheng, Sijia Cai, Yuan Liu, Bing Deng, Jianqiang Huang, Xiansheng Hua, Mingbi Zhao. IEEE International Conference on Computer Vision, 2021

306 citations recorded by the source corpus.

Though 3D object detection from point clouds has achieved rapid progress in recent years, the lack of flexible and high-performance proposal refinement remains a great hurdle for existing state-of-the-art two-stage detectors. Previous works on refining 3D proposals have relied on human-designed components such as keypoints sampling, set abstraction and multi-scale feature fusion to produce powerful 3D object representations. Such methods, however, have limited ability to capture rich contextual dependencies among points. In this paper, we leverage the high-quality region proposal network and a Channel-wise Transformer architecture to constitute our two-stage 3D object detection framework (CT3D) with minimal hand-crafted design. The proposed CT3D simultaneously performs proposal-aware embedding and channel-wise context aggregation for the point features within each proposal. Specifically, CT3D uses proposal’s keypoints for spatial contextual modelling and learns attention propagation in the encoding module, mapping the proposal to point embeddings. Next, a new channel-wise decoding module enriches the query-key interaction via channel-wise re-weighting to effectively merge multi-level contexts, which contributes to more accurate object predictions. Extensive experiments demonstrate that our CT3D method has superior performance and excellent scalability. Remarkably, CT3D achieves the AP of 81.77% in the moderate car category on the KITTI test 3D detection benchmark, outperforms state-of-the-art 3D detectors.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Industrial and Manufacturing Engineering

View Paper: https://www.semanticscholar.org/paper/312160aa805e1c9936b26359c7c2e0acef81b936

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Industrial and Manufacturing Engineering

Citations in the corpus

Point2Seq: Detecting 3D Objects as Sequences (2022, unknown): “Our approach even outperforms those time-consuming two-stage 3D detectors [16, 24, 25], which indicates the effectiveness of the scene-to-sequence decoder.”
TransFuser: Imitation With Transformer-Based Sensor Fusion for Autonomous Driving (2022, background): “Attention has been explored in the context of driving for lane changing [65], object detection [31], [66], [67], [68], motion forecasting [31], [69], [70], [71], [72], [73], [74], [75], [76], [77], driver attention prediction [78], [79] and recently also for end-to-end driving [39], [41].”
3D Object Detection for Autonomous Driving: A Survey (2021, methodology): “+ TANet [76], AAAI′20 + VeloFCN [48], RSS′16 + SPG [77], ICCV′21 + PIXOR [49], CVPR′18 + CenterPoint [23], CVPR′21 + VoxelNet [24], CVPR′18 + BtcDet [10], AAAI′22 + SECOND [25], Sensors′18 + PointPillars [6], CVPR′19 + CIA-SSD [26], AAAI′21 + SE-SSD [27], CVPR′21 + Voxel R-CNN [28], AAAI′21 + CT3D [”
PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection (2021, background): “Most of state-of-the-art 3D detectors (Shi et al., 2020b; Yin et al., 2021; Sheng et al., 2021) adopt 3D sparse convolution for learning representative features from irregular points thanks to its efﬁciency and effectiveness on handling large-scale point clouds.”
A Survey of Visual Transformers (2021, methodology): “Sheng et al. [118] proposed a typical two-stage method that leverages a chan-nelwise transformer 3-”
TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers (2022, methodology): “Despite the popularity of adopting the transformer architecture as a detection head in 2D [2], 3D detection models for outdoor scenarios mostly utilize the transformer for feature extraction [21, 24, 35].”
Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Clouds (2022, unknown): “In order to compensate for the structural distortion in the preprocessing phase, complex module design needs to be introduced in [20, 25, 27, 28, 35], which in turn greatly deteriorate the final detection efficiency.”