Holistic Scene Understanding for 3D Object Detection with RGBD Cameras

By Dahua Lin, Sanja Fidler, Raquel Urtasun. 2013

304 citations recorded by the source corpus.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

Citations in the corpus

3D fully convolutional network for vehicle detection in point cloud (2016, methodology): “[3], [9], [16], [17], [24], [25] embed 3D information in 2D projection and use 2D CNN for recognition or detection.”
Vehicle Detection from 3D Lidar Using Fully Convolutional Network (2016, background): “We would also like to mention that object detection on RGBD images [3, 17] is closely related to the topic of object detection on range scan.”
The limits and potentials of deep learning for robotics (2018, methodology): “The work by Lin et al. [65] is an example of a scene understanding approach that explicitly models and exploits several semantic and geometric relations between objects and the overall scene using Conditional Random Fields.”
A Survey of Autonomous Driving: Common Practices and Emerging Technologies (2019, methodology): “point cloud data in 2D, the first being a so-called depth image shown in Figure 14, largely inspired by camera-based methods that perform 3D object detection through depth estimation [165] and methods that operate on RGB-D data [166].”
Semantic Scene Completion from a Single Depth Image (2016, methodology): “Alternatively, it is possible to use 3D primitives such as bounding boxes to approximate the 3D geometry of objects [11, 18, 31].”
"What Happens If..." Learning to Predict the Effect of Forces in Images (2016, background): “There is a considerable body of work on scene understanding in the vision literature, e.g., [26,9,41,21,13,32,4,22,43].”
Scene Graph Generation by Iterative Message Passing (2017, background): “A series of contextual models based on surrounding pixels and regions have also been developed for perceptual tasks [3, 13, 25, 27].”
Matterport3D: Learning from RGB-D Data in Indoor Environments (2017, methodology): “These previous RGB-D datasets have been used to train models for several standard scene understanding tasks, including semantic segmentation [27, 33, 13, 40, 7], 3D object detection [31, 24, 14, 36, 37], normal estimation [41, 23, 9, 3], camera relocalization [30, 39], and others [47, 11, 12].”
Towards a Visual Turing Challenge (2014, methodology): “The current state-of-the-art semantic segmentation methods on the NYU-Depth V2 dataset [42] can discriminate only between up to 37 object categories [2, 52, 53], much fewer to what is needed.”
Deep Hough Voting for 3D Object Detection in Point Clouds (2019, methodology): “Examples include: [27] where a pair-wise semantic context potential helps guide the proposals objectness score; templatebased methods [26, 32, 28]; Sliding-shapes [41] and its deep learning-based successor [42]; Clouds of Oriented Gradients (COG) [38]; and the recent 3D-SIS [12].”
DeepContext: Context-Encoding Neural Pathways for 3D Holistic Scene Understanding (2016, background): “While most existing research is limited to 2D, there are some works on modeling context for total scene understanding from RGBD images [36, 28, 46, 15, 23].”
Three-Dimensional Object Detection and Layout Prediction Using Clouds of Oriented Gradients (2016, methodology): “Other work has used point cloud data to directly predict 3D layout [25, 35], but can be sensitive to errors in RGB-D depth estimates.”
The Overlooked Elephant of Object Detection: Open Set (2020, background): “…+ β 2 precision Since F β is just a combination of the precision and recall values and does not need a value for true negatives, which are needed by metrics such as accuracy and are not available for a detection problem, it naturally has become the second choice for the detection community [14, 1].”
Joint 3D Object and Layout Inference from a Single RGB-D Image (2015, methodology): “While existing approaches to the 3D scene understanding problem typically infer only objects [16, 17] or consider layout estimation as a pre-processing step [25], our method reasons jointly about 3D objects and the scene layout.”
Learning Rich Features from RGB-D Images for Object Detection and Segmentation (2014, methodology): “[28] with and without non-maximum suppression, MCG with RGB contours and RGB features, MCG with RGB-D contours but RGB features and finally our system which is MCG with RGB-D contours and RGB-D features.”
Monocular 3D Object Detection for Autonomous Driving (2016, background): “In RGB-D, [10] exploited stereo imagery to exhaustively scored 3D bounding boxes using a conditional random field with several depth-informed potentials.”
Resolving Language and Vision Ambiguities Together: Joint Segmentation & Prepositional Attachment Resolution in Captioned Scenes (2016, background): “In Section 4, we describe each of the modules in detail, making concrete their respective features, and other details.”
3D Object Proposals for Accurate Object Class Detection (2015, background): “[17, 18] extend CPMC [10] with additional affinities that encourage the proposals to respect occlusion boundaries.”