Learning Depth-Guided Convolutions for Monocular 3D Object Detection

By Mingyu Ding, Yuqi Huo, Hongwei Yi, Zhe Wang, Jianping Shi, Zhiwu Lu, Ping Luo. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2019

372 citations recorded by the source corpus.

3D object detection from a single image without LiDAR is a challenging task due to the lack of accurate depth information. Conventional 2D convolutions are unsuitable for this task because they fail to capture local object and its scale information, which are vital for 3D object detection. To better represent 3D structure, prior arts typically transform depth maps estimated from 2D images into a pseudo-LiDAR representation, and then apply existing 3D point-cloud based object detectors. However, their results depend heavily on the accuracy of the estimated depth maps, resulting in suboptimal performance. In this work, instead of using pseudo-LiDAR representation, we improve the fundamental 2D fully convolutions by proposing a new local convolutional network (LCN), termed Depth-guided Dynamic-Depthwise-Dilated LCN (D4LCN), where the filters and their receptive fields can be automatically learned from image-based depth maps, making different pixels of different images have different filters. D4LCN overcomes the limitation of conventional 2D convolutions and narrows the gap between image representation and 3D point cloud representation. Extensive experiments show that D4LCN outperforms existing works by large margins. For example, the relative improvement of D4LCN against the state-of-the-art on KITTI is 9.1% in the moderate setting. D4LCN ranks 1st on KITTI monocular 3D object detection benchmark at the time of submission (car, December 2019). The code is available at https://github.com/dingmyu/D4LCN.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/8841036470c24823c2f33dcfbff855f12761876e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Domain-Adaptive Few-Shot Learning (2020, background): “Unlike many visual recognition methods [17, 34, 60, 7, 1, 6, 8] that require a lot of supervision, few-shot learning (FSL) [9, 28, 23, 27, 16] aims to recognize a set of target”
3D Object Detection for Autonomous Driving: A Comprehensive Survey (2022, background): “a point-based attentional network [7], depth-guided convolutions [60], depth-”
Geometry Uncertainty Projection Network for Monocular 3D Object Detection (2021, background): “The monocular 3D object detection aims to predict 3D bounding boxes from a single given image [13, 17, 21, 25, 31, 43].”
BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection (2022, methodology): “D(4)LCN [9] employs depth maps to guide dynamic kernel learning.”
Spatial Information Guided Convolution for Real-Time RGBD Semantic Segmentation (2020, background): “[43] proposes to perform 3D object detection based on depth-guided convolution, whose weights are location-variant and depth-adaptive.”
Categorical Depth Distribution Network for Monocular 3D Object Detection (2021, methodology): “To combat this effect, monocular object detection methods [13, 36, 37, 59] often learn depth explicitly, by training a monocular depth estimation network in a separate stage.”
Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Language (2021, background): “Physical Scene Understanding Our work is also relevant to studies on physical scene understanding [7, 73, 66, 64, 8, 25, 26, 20, 50, 39], most of which propose pure neural-network solutions without explicitly incorporating physics models.”
FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection (2021, background): “Others [1, 9, 2] simultaneously predict 2D and 3D boxes with 3D priors corresponding to each 2D anchor.”
Delving into Localization Errors for Monocular 3D Object Detection (2021, methodology): “In contrast, the accuracy of 3D detection from only monocular images [35, 2, 10, 25, 24, 12] is obviously lower than that from LiDAR or stereo.”
Objects are Different: Flexible Monocular 3D Object Detection (2021, methodology): “*Corresponding author (a) M3D-RPN [3] (b) D4LCN [13]”
Is Pseudo-Lidar needed for Monocular 3D Object detection? (2021, methodology): “The use of depth-aware convolution or dense 3D anchors [10, 53] enabled higher accuracy.”