M3D-RPN: Monocular 3D Region Proposal Network for Object Detection

By Garrick Brazil, Xiaoming Liu. IEEE International Conference on Computer Vision, 2019

556 citations recorded by the source corpus.

Understanding the world in 3D is a critical component of urban autonomous driving. Generally, the combination of expensive LiDAR sensors and stereo RGB imaging has been paramount for successful 3D object detection algorithms, whereas monocular image-only methods experience drastically reduced performance. We propose to reduce the gap by reformulating the monocular 3D detection problem as a standalone 3D region proposal network. We leverage the geometric relationship of 2D and 3D perspectives, allowing 3D boxes to utilize well-known and powerful convolutional features generated in the image-space. To help address the strenuous 3D parameter estimations, we further design depth-aware convolutional layers which enable location specific feature development and in consequence improved 3D scene understanding. Compared to prior work in monocular 3D detection, our method consists of only the proposed 3D region proposal network rather than relying on external networks, data, or multiple stages. M3D-RPN is able to significantly improve the performance of both monocular 3D Object Detection and Bird's Eye View tasks within the KITTI urban autonomous driving dataset, while efficiently using a shared multi-class model.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/7de7b708089b1cd9c3cbdaf58ac208e9430cd9a3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Probabilistic and Geometric Depth: Detecting Objects in Perspective (2021, methodology): “Our framework based on the simple adapted FCOS3D achieves much better performance than others, especially considering M3D-RPN [13] and RTM3D [14] adopt stronger backbone and data augmentation.”
RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving (2020, methodology): “25 M3D-RPN[4] 0.”
Is Pseudo-Lidar needed for Monocular 3D Object detection? (2021, background): “While the loss in accuracy between KITTI-3D val and the test sets is consistent with other methods [57, 4, 3], PL suffers particularly more (30.”
PETR: Position Embedding Transformation for Multi-View 3D Object Detection (2022, background): “M3D-RPN [2] introduces the depth-aware convolution, which learns position-aware features for 3D object detection.”
Geometry Uncertainty Projection Network for Monocular 3D Object Detection (2021, background): “M3DRPN [4] gave a novel modified 3D anchor setting and proposed a depthwise convolution to treat the monocular 3D detection task.”
Unifying Voxel-based Representation with Transformer for 3D Object Detection (2022, methodology): “With monocular image, previous approaches try to predict 3D boxes based on image features directly [25, 26, 27] or utilize the middle representation [1, 2, 5].”
Learning Depth-Guided Convolutions for Monocular 3D Object Detection (2019, methodology): “(2) Most competitors, such as [23, 33, 45, 37, 50, 1], utilize the detector (e.”
Delving into Localization Errors for Monocular 3D Object Detection (2021, methodology): “Here we briefly review the ‘standard’ monocular 3D detection approaches [27, 32, 29, 35, 2, 43] only use the RGB images, annotations and camera calibrations provided by KITTI dataset.”
BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection (2022, background): “M3DRPN [8] proposes depth-aware convolutional layers to enhance spatial awareness.”
FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection (2021, methodology): “For example, M3D-RPN [1] proposes a single-stage detector with an end-to-end region proposal network and depth-aware convolution.”
SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimation (2020, methodology): “The only one-stage method M3D-RPN [1] proposes a standalone network to generate 2D and 3D object proposals simultaneously.”
BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers (2022, methodology): “For the 3D object detection task, early methods are similar to 2D detection methods [1, 28, 48, 39, 52], which usually predict the 3D bounding boxes based on 2D bounding boxes.”
Objects are Different: Flexible Monocular 3D Object Detection (2021, methodology): “Figure 1: Qualitative comparison among prior arts [3, 13], our baseline, and the proposed method.”
PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection (2021, methodology): “(Liu et al., 2019) combines both voxel CNN and point multi-layer percetron network for efﬁcient point feature learning.”
PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images (2022, unknown): “Previous works [2,6,13–15, 29, 35 , 37, 38 ] perform 3D object object”
Categorical Depth Distribution Network for Monocular 3D Object Detection (2021, methodology): “M3D-RPN [3] introduces depth-aware convolutions that divides the input row-wise and learns non-shared kernels for each region, to learn location specific features that correlate to regions in 3D space.”
MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationships (2020, methodology): “Therefore, there is a growing trend on performing monocular 3D object detection in research community in recent years [3, 5, 26, 27, 31, 36].”