Stereo R-CNN Based 3D Object Detection for Autonomous Driving

By Peiliang Li, Xiaozhi Chen, Shaojie Shen. Computer Vision and Pattern Recognition, 2019

565 citations recorded by the source corpus.

We propose a 3D object detection method for autonomous driving by fully exploiting the sparse and dense, semantic and geometry information in stereo imagery. Our method, called Stereo R-CNN, extends Faster R-CNN for stereo inputs to simultaneously detect and associate object in left and right images. We add extra branches after stereo Region Proposal Network (RPN) to predict sparse keypoints, viewpoints, and object dimensions, which are combined with 2D left-right boxes to calculate a coarse 3D object bounding box. We then recover the accurate 3D bounding box by a region-based photometric alignment using left and right RoIs. Our method does not require depth input and 3D position supervision, however, outperforms all existing fully supervised image-based methods. Experiments on the challenging KITTI dataset show that our method outperforms the state-of-the-art stereo-based method by around 30% AP on both 3D detection and 3D localization tasks. Code will be made publicly available.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/da4656d88d3363488219682fe2c5dacfbb729480

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

EPNet: Enhancing Point Features with Image Semantics for 3D Object Detection (2020, background): “Recent 3D object detection methods pay much attention to camera images, such as monocular [23,29,12,15,20] and stereo images [16,35].”
FVNet: 3D Front-View Proposal Generation for Real-Time Object Detection from Point Clouds (2019, methodology): “3D object detection [3, 4, 10, 32, 24, 13] has become increasingly important in many fields, such as autonomous driving [1], mobile robots [27] and virtual/augmented reality [17].”
Adversarial Defense Via Local Flatness Regularization (2019, unknown): “Deep neural networks (DNNs) have been widely and successfully used in many computer vision area, such as pose estimation [1, 2, 3], object detection [4, 5, 6] and super-resolution [7, 8, 9].”
SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimation (2020, background): “Stereo R-CNN [16] creates 2D proposals simultaneously on stereo images.”
Classification of Hyperspectral Image Based on Double-Branch Dual-Attention Mechanism Network (2019, background): “significant breakthroughs, such as objection detection [18], natural language processing [19], and image classification [20].”
MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationships (2020, methodology): “Most existing methods on 3D object detection require accurate depth information, which can be obtained from either 3D LiDARs [8, 30, 34, 35, 23, 45] or multicamera systems [6, 7, 20, 29, 32, 41].”
Disentangling Monocular 3D Object Detection (2019, methodology): “n generated 3D hypotheses and corresponding, 2D pseudomeasurements. LIDARand/orstereo-based. 3DOP [5] exploits stereo images and prior knowledge about the scene to directly reason in 3D. Stereo R-CNN [16] tackles 3D object detection by exploiting stereo imagery and produces stereo boxes, keypoints, ”
From Points to Parts: 3D Object Detection From Point Cloud With Part-Aware and Part-Aggregation Network (2019, background): “Recently [43], [44] explored the stereo pair of images to improve the 3D detection performance from stereo cameras.”
RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving (2020, methodology): “Another one only employs 2D bounding box and properties of a 3D object as the supervised data [35, 4, 22, 44].”
Multimodal Virtual Point 3D Detection (2021, methodology): “However, they rely on noisy stereo depth estimates [25, 40, 56] while we use more accurate Lidar measurements.”
PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection (2021, methodology): “Fast Point R-CNN (Chen et al., 2019b) fuses the deep voxel features with raw points for 3D detection.”
F-cooper: feature based cooperative perception for autonomous vehicle edge computing system using 3D point clouds (2019, background): “Due to the popularity of CNN based solutions to object detection for autonomous driving vehicles [4, 15, 25], in this paper, we focus on”
Probabilistic and Geometric Depth: Detecting Objects in Perspective (2021, methodology): “Recent advanced methods in this field typically resort to various sensors, such as LiDAR [1, 2, 3, 4, 5, 6], Radar [7], binocular vision [8, 9], or their combinations for accurate depth information.”