Disentangling Monocular 3D Object Detection

By Andrea Simonelli, Samuel Rota Bulò, Lorenzo Porzi, Manuel López-Antequera, Peter Kontschieder. IEEE International Conference on Computer Vision, 2019

511 citations recorded by the source corpus.

In this paper we propose an approach for monocular 3D object detection from a single RGB image, which leverages a novel disentangling transformation for 2D and 3D detection losses and a novel, self-supervised confidence score for 3D bounding boxes. Our proposed loss disentanglement has the twofold advantage of simplifying the training dynamics in the presence of losses with complex interactions of parameters, and sidestepping the issue of balancing independent regression terms. Our solution overcomes these issues by isolating the contribution made by groups of parameters to a given loss, without changing its nature. We further apply loss disentanglement to another novel, signed Intersection-over-Union criterion-driven loss for improving 2D detection results. Besides our methodological innovations, we critically review the AP metric used in KITTI3D, which emerged as the most important dataset for comparing 3D detection results. We identify and resolve a flaw in the 11-point interpolated AP metric, affecting all previously published detection results and particularly biases the results of monocular 3D detection. We provide extensive experimental evaluations and ablation studies and set a new state-of-the-art on the KITTI3D Car class.

Subfields: Computer Vision and Pattern Recognition, Industrial and Manufacturing Engineering

View Paper: https://www.semanticscholar.org/paper/9e08fff0a66020bc72bd48685f87d21d2ff78d66

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Industrial and Manufacturing Engineering

Citations in the corpus

Probabilistic and Geometric Depth: Detecting Objects in Perspective (2021, methodology): “MonoDIS [12] introduces a disentangling loss to reduce the instability of the training procedure.”
CosyPose: Consistent multi-view multi-object 6D pose estimation (2020, methodology): “Third, we disentangle depth and translation prediction in the loss following [42] and handle symmetries explicitly as in [9] instead of using the point-matching loss.”
Objects are Different: Flexible Monocular 3D Object Detection (2021, background): “As discussed in [36, 42], the separate optimization of multiple subtasks cannot ensure the optimal cooperation among different components.”
PointPainting: Sequential Fusion for 3D Object Detection (2019, methodology): “ore, to establish the generality of PointPainting, we chose to submit Painted PointPillars results to nuScenes test, and use our KITTI submission on Painted PointRCNN. Method Modality NDS mAP MonoDis [23] Images 38.4 30.4 PointPillars [11,1] Lidar 45.3 30.5 PointPillars+ Lidar 55.0 40.1 Painted Poin”
SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimation (2020, methodology): “Following prior work [22, 31], the orientation can be encoded as a 4D quaternion to formulate 3D bounding box.”
FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection (2021, background): “Some previous work [30, 20] keeps predicting 2D boxes and further regresses 3D attributes on top of 2D centers and regions of interest.”
Tracking Objects as Points (2020, methodology): “ion on the KITTI test set. We compare to all published entries on the leaderboard. Runtimes are from the leaderboard. +D means detection time. Time(ms) AMOTA@0.2 "AMOTA@1 "AMOTP # Mapillary [38]+AB3D [46] - 6.9 1.8 1.8 Ours 45 27.8 4.6 1.5 Table 3: Evaluation on the nuScenes test set. We c”
Is Pseudo-Lidar needed for Monocular 3D Object detection? (2021, methodology): “In [57], the authors proposed a disentangled loss for 3D box regression that helps stabilize training.”
Geometry Uncertainty Projection Network for Monocular 3D Object Detection (2021, methodology): “The monocular 3D object detection aims to predict 3D bounding boxes from a single given image [13, 17, 21, 25, 31, 43].”
RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving (2020, unknown): “Mono3D++ [13],Deep-MANTA [5], 3DVP [38] X Deep3DBox [26],GS3D [19],MonoGRNet [31], FQNet[23], M3D-RPN[3] Shift-RCNN [28], MonoDIS [34] Ours(RTM3D) X”
PETR: Position Embedding Transformation for Multi-View 3D Object Detection (2022, background): “Many previous works [6,33,20,21,41,19,2,49,48] perform 3D object detection in the image view.”
nuScenes: A Multimodal Dataset for Autonomous Driving (2019, methodology): “PP: PointPillars [51], MDIS: MonoDIS [70].”
BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers (2022, methodology): “For the 3D object detection task, early methods are similar to 2D detection methods [1, 28, 48, 39, 52], which usually predict the 3D bounding boxes based on 2D bounding boxes.”
BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervision (2022, methodology): “MonoDIS [31] used RoIAlign to extract image features from 2D boxes and to regress 3D boxes subsequently.”
Categorical Depth Distribution Network for Monocular 3D Object Detection (2021, methodology): “Direct methods [9, 52, 4, 32] estimate 3D detections directly from images without predicting an intermediate 3D scene representation.”
Track to Detect and Segment: An Online Multi-Object Tracker (2021, unknown): “66 2,252 37ms Mapillary[34]+AB3D[47] 12.”
PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images (2022, background): “Previous works [4, 26, 13, 14, 33, 12, 1, 36, 35] perform 3D object detection mainly under the mono setting.”
Structure Aware Single-Stage 3D Object Detection From Point Cloud (2020, background): “2019, KITTI changed its evaluation setting by using 40 recall positions as suggested in [21].”