GS3D: An Efficient 3D Object Detection Framework for Autonomous Driving

By Buyu Li, Wanli Ouyang, Lu Sheng, Xingyu Zeng, Xiaogang Wang. Computer Vision and Pattern Recognition, 2019

351 citations recorded by the source corpus.

We present an efficient 3D object detection framework based on a single RGB image in the scenario of autonomous driving. Our efforts are put on extracting the underlying 3D information in a 2D image and determining the accurate 3D bounding box of object without point cloud or stereo data. Leveraging the off-the-shelf 2D object detector, we propose an artful approach to efficiently obtain a coarse cuboid for each predicted 2D box. The coarse cuboid has enough accuracy to guide us to determine the 3D box of the object by refinement. In contrast to previous state-of-the-art methods that only use the features extracted from the 2D bounding box for box refinement, we explore the 3D structure information of the object by employing the visual features of visible surfaces. The new features from surfaces are utilized to eliminate the problem of representation ambiguity brought by only using 2D bounding box. Moreover, we investigate different methods of 3D box refinement and discover that a classification formulation with quality aware loss have much better performance than regression. Evaluated on KITTI benchmark, our approach outperforms current state-of-the-art methods for single RGB image based 3D object detection.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/d19356ce442d9c04625b3a253f370feaf00ea6a0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Accurate Monocular 3D Object Detection via Color-Embedded 3D Reconstruction for Autonomous Driving (2019, background): “In contrast, some other studies [1, 4, 3, 37, 23, 35, 15] are devoted to replace the LiDAR with cheaper monocular cameras, which are readily available in daily life.”
PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection (2021, background): “The following works [42], [43] incorporate the relations between 2D and 3D boxes as geometric constraint.”
Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection (2020, methodology): “As motivated by (Jiang et al. 2018; Li et al. 2019; Shi et al. 2020a,b), the box regression branch predicts the residue from 3D region proposals to the ground truth boxes, and the conﬁdence branch predicts the IoU-related conﬁdence score.”
Learning Depth-Guided Convolutions for Monocular 3D Object Detection (2019, methodology): “Recent methods towards the above goal can be generally categorized into two streams as image-based approaches [36, 26, 41, 19, 17, 4] and pseudo-LiDAR point-based approaches [48, 33, 50].”
EPNet: Enhancing Point Features with Image Semantics for 3D Object Detection (2020, background): “Recent 3D object detection methods pay much attention to camera images, such as monocular [23,29,12,15,20] and stereo images [16,35].”
PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection (2019, methodology): “For the confidence prediction branch, we follow [15, 9, 28] to adopt the 3D Intersection-over-Union (IoU) between the 3D RoIs and their corresponding ground-truth boxes as the training targets.”
From Points to Parts: 3D Object Detection From Point Cloud With Part-Aware and Part-Aggregation Network (2019, methodology): “[34], [35] leveraged the geometry constraints between 3D and 2D bounding box to recover the 3D object pose.”
Geometry Uncertainty Projection Network for Monocular 3D Object Detection (2021, methodology): “GS3D [22] utilized the ROI surface features to extract better object representations.”
PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation (2020, methodology): “The contextual and geometric information is well extracted by the U-Net, which provides discriminative point-wise features F for subsequent processing.”
Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving (2019, neutral): “…built on the 2D frontal-view detection pipeline (Ren et al., 2015; He et al., 2017; Lin et al., 2017) — fell far behind in localizing objects in 3D (Li et al., 2019a; Xiang et al., 2015; 2017; Chabot et al., 2017; Mousavian et al., 2017; Chen et al., 2015; Xu & Chen, 2018; Chen et al., 2016;…”
Center-based 3D Object Detection and Tracking (2020, methodology): “For class-agnostic confidence score prediction, we follow [25, 29, 42, 44] and use a score target I guided by the box’s 3D IoU with the corresponding ground truth bounding box:”
Is Pseudo-Lidar needed for Monocular 3D Object detection? (2021, methodology): “Other methods try to leverage geometric consistency between 2D and 3D structures, formulating the inference as a constrained optimization problem [45, 38, 46, 12, 44, 24, 27, 34].”
PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud (2018, background): “[24, 15] leveraged the geometry constraints between 3D and 2D bounding box to recover the 3D object pose.”
YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors (2022, background): “For example, multi-object tracking [94, 93], autonomous driving [40, 18], robotics [35, 58], medical image analysis [34, 46], etc.”
RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving (2020, neutral): “…Depth Shape/CAD Segmentation Mono3D [7] 3DOP [8], stereoRCNN [22] MF3D [42], Peseudo-LiDAR [39], MonoPSR [17] AM3D[27] Mono3D++ [14],Deep-MENTA [6], 3DVP [40] Deep3DBox [28],GS3D [20],MonoGRNet [33], FQNet[25], M3D-RPN[4] Shift-RCNN [30] Ours(RTM3D) Image-only in Monocular 3D Object Detection.”