Categorical Depth Distribution Network for Monocular 3D Object Detection

By Cody Reading, Ali Harakeh, Julia Chae, Steven L. Waslander. Computer Vision and Pattern Recognition, 2021

641 citations recorded by the source corpus.

Monocular 3D object detection is a key problem for autonomous vehicles, as it provides a solution with simple configuration compared to typical multi-sensor systems. The main challenge in monocular 3D detection lies in accurately predicting object depth, which must be inferred from object and scene cues due to the lack of direct range measurement. Many methods attempt to directly estimate depth to assist in 3D detection, but show limited performance as a result of depth inaccuracy. Our proposed solution, Categorical Depth Distribution Network (CaDDN), uses a predicted categorical depth distribution for each pixel to project rich contextual feature information to the appropriate depth interval in 3D space. We then use the computationally efficient bird’s-eye-view projection and single-stage detector to produce the final output detections. We design CaDDN as a fully differentiable end-to-end approach for joint depth estimation and object detection. We validate our approach on the KITTI 3D object detection benchmark, where we rank 1st among published monocular methods. We also provide the first monocular 3D detection results on the newly released Waymo Open Dataset. We provide a code release for CaDDN which is made available.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/bd519be5f86187eb98ab1234b60afe94949bbca5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers (2022, methodology): “BEVFormer outperforms CaDNN with APH of 13.3% and 11.2% on LEVEL_1 and LEVEL_2 difﬁculties with IoU criteria of 0.5.”
BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework (2022, methodology): “Note that, people discover that bird’s eye view (BEV) has been an de-facto standard for autonomous driving scenarios as, generally speaking, car cannot fly [19, 22, 37, 15, 52, 31].”
PETR: Position Embedding Transformation for Multi-View 3D Object Detection (2022, methodology): “Similar to CaDDN [38], BEVDet [18] employs the Lift-Splat-Shoot [37] to transform 2D multi-view features into BEV representation.”
BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection (2022, methodology): “LSS [3] proposes a view transform method that explicitly predicts depth distribution and projects image features onto bird’s-eye view (BEV), which has been proved practical for 3D object detection [2, 1, 17].”
DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection (2022, background): “The success of deep neural networks brings the promise of solving AD’s core requirement to perceive the surrounding environment from point cloud [15,20,28], images [7, 18] or multi-modality data [21, 24].”
Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps (2022, methodology): “For camera-only 3D object detection task on OPV2V, we implement the detector following CADDN [1].”
BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection (2022, background): “In the last few years, fueled by the KITTI [11] benchmark monocular 3D object detection has witness a rapid development [26, 23, 47, 53, 49, 31, 39, 38, 16].”
Unifying Voxel-based Representation with Transformer for 3D Object Detection (2022, methodology): “Motivated by [39, 5], we first generate the depth distribution DI ∈ RD×H×W of each image with a single convolution as DI(u, v) = Softmax(Conv(FI)(u, v)).”
OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Prediction (2023, methodology): “Considering the dimension gap between the 2D image input and the 3D prediction, recent studies for vision-based 3D perception first construct the BEV feature representations and then perform various downstream tasks on the BEV space [19, 28, 30, 37, 58, 38, 60, 40, 18, 1, 42].”
V2V4Real: A Real-World Large-Scale Dataset for Vehicle-to-Vehicle Cooperative Perception (2023, background): “(1) Camerabased detection denotes approaches that detect 3D objects from a single or multiple RGB images [16, 32, 33, 35, 42].”
Multimodal Virtual Point 3D Detection (2021, background): “On state-of-the-art 3D detection benchmarks [2, 12], state-of-the-art monocular methods [26, 41] achieve about half the mAP detection accuracy, of standard Lidar based baselines [60].”
BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation (2022, background): “A widespread of various depth-depended tasks related to autonomous driving [2], [3], [4], [5], [6], virtual reality [7], [8], and scene understanding [9], [10], [11], [12] provide strong demand for effective MDE methods that can accurately reconstruct the 3D world.”
PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection (2021, background): “LSS (Philion and Fidler, 2020) andCaDDN (Reading et al., 2021) predicts depth distribution to “lift” the 2D image features to a BEV feature map for 3D detection.”
3D Object Detection for Autonomous Driving: A Survey (2021, methodology): “The other line is to directly lift 2D image features into 3D space via computing a point cloud [47, 45, 41, 71, 73, 75], termed as Pseudo LiDAR or learning a latent depth distribution [4, 72, 74, 70].”
SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving (2023, methodology): “Depth-based methods explicitly predict depth maps for image inputs to extract 3D geometric information of the scene before perception [54, 36, 44, 39, 28, 35, 30, 42, 21, 65].”
BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View (2021, background): “Among them, also inspired by Lift-Splat-Shoot [30], [32] is the most similar one as ours.”
Geometry Uncertainty Projection Network for Monocular 3D Object Detection (2021, methodology): “For instance, compared with the recently proposed CaDNN [38] utilizing LiDAR signals as supervision of depth estimation sub-task, our method still obtains 0.”
Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction (2023, background): “Recent studies in BEV-based perception focus on how to effectively transform features from the image space to the BEV space [20, 26, 27, 35, 36, 48].”