FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection

By Tai Wang, Xinge Zhu, Jiangmiao Pang, Dahua Lin. 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2021

814 citations recorded by the source corpus.

Monocular 3D object detection is an important task for autonomous driving considering its advantage of low cost. It is much more challenging than conventional 2D cases due to its inherent ill-posed property, which is mainly reflected in the lack of depth information. Recent progress on 2D detection offers opportunities to better solving this problem. However, it is non-trivial to make a general adapted 2D detector work in this 3D task. In this paper, we study this problem with a practice built on a fully convolutional single-stage detector and propose a general framework FCOS3D. Specifically, we first transform the commonly defined 7-DoF 3D targets to the image domain and decouple them as 2D and 3D attributes. Then the objects are distributed to different feature levels with consideration of their 2D scales and assigned only according to the projected 3D-center for the training procedure. Furthermore, the center-ness is redefined with a 2D Gaussian distribution based on the 3D-center to fit the 3D target formulation. All of these make this framework simple yet effective, getting rid of any 2D detection or 2D-3D correspondence priors. Our solution achieves 1st place out of all the vision-only methods in the nuScenes 3D detection challenge of NeurIPS 2020. Code and models are released at https://github.com/open-mmlab/mmdetection3d.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/993e389503ad2e4a473b5b0db3abe90f094c811d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving (2023, methodology): “For nuScenes dataset, we adopt ResNet101-DCN [18, 13] with the initial weight from FCOS3D [54] as the backbone to extract image features.”
FUTR3D: A Unified Sensor Fusion Framework for 3D Detection (2022, methodology): “Modality NDS ↑ mAP ↑ mATE ↓ mASE ↓ mAOE ↓ mAVE ↓ mAAE ↓ FCOS3D [29] C 40.”
Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving (2023, neutral): “More recently, vision-based 3D object detection has gained more attention due to its low cost and rich semantic content [41, 46, 48, 22, 26, 33, 28, 13, 27, 15, 21, 31, 25].”
BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers (2022, methodology): “Method Modality Backbone NDS↑ mAP↑ mATE↓ mASE↓ mAOE↓ mAVE↓ mAAE↓ FCOS3D [45] C R101 0.”
PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images (2022, methodology): “Previous works [4, 26, 13, 14, 33, 12, 1, 36, 35] perform 3D object detection mainly under the mono setting.”
BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation (2022, background): “FCOS3D [30] extends an image detector [31] with additional 3D regression branches, which is later improved by [32], [33] in depth modeling.”
BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View (2021, methodology): “For monocular paradigms like FCOS3D [42] and PGD [43], the inference speeds are divided by a factor of 6 (i.”
Unifying Voxel-based Representation with Transformer for 3D Object Detection (2022, methodology): “With monocular image, previous approaches try to predict 3D boxes based on image features directly [25, 26, 27] or utilize the middle representation [1, 2, 5].”
Probabilistic and Geometric Depth: Detecting Objects in Perspective (2021, methodology): “For simplicity, we take FCOS3D [15], the 3D adapted version of FCOS [24], as the baseline considering its capability of handling overlapped ground truths and scale variance problem.”
BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection (2022, methodology): “• The speed of depth-based 3D detectors is empirically slower than depth-free detectors like FCOS3D [5] when using the same image resolution and backbone, because the viewtransform sub-network that comes after depth estimation suffers from extremely poor efficiency.”
BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervision (2022, methodology): “MV-FCOS3D++ [32] introduced perspective super-vision for training its image backbone, but the detector itself was supervised by BEV losses alone.”
PETR: Position Embedding Transformation for Multi-View 3D Object Detection (2022, methodology): “Many previous works [6,33,20,21,41,19,2,49,48] perform 3D object detection in the image view.”
BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework (2022, methodology): “With the development of autonomous driving datasets that have more sensors, like nuScenes[2] and Waymo [44], there exists a trend of developing methods [50, 51, 53] that take multi-view images as input and found to be signiﬁcantly superior to monocular methods.”
BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection (2022, methodology): “Thus, the existing paradigms [40, 41, 42, 22, 15] perform relatively poorly in predicting the time-relevant targets like velocity than the LiDAR-based [46] or radar-based [27] methods.”
Is Pseudo-Lidar needed for Monocular 3D Object detection? (2021, methodology): “DD3D outperforms all other methods with a 17% improvement in mAP compared to the previously best published method [63] (41.”
Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction (2023, methodology): “TPVFormer-Base uses the ResNet101DCN [15, 19] initialized from FCOS3D [44] checkpoint, while TPVFormer-Small adopts the ResNet-50 [19] pretrained on ImageNet [16].”
DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries (2021, methodology): “For details, see [2].”