Geometric reasoning for single image structure recovery

By David C. Lee, Martial Hebert, Takeo Kanade. 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009

489 citations recorded by the source corpus.

We study the problem of generating plausible interpretations of a scene from a collection of line segments automatically extracted from a single indoor image. We show that we can recognize the three dimensional structure of the interior of a building, even in the presence of occluding objects. Several physically valid structure hypotheses are proposed by geometric reasoning and verified to find the best fitting model to line segments, which is then converted to a full 3D model. Our experiments demonstrate that our structure recovery from line segments is comparable with methods using full image appearance. Our approach shows how a set of rules describing geometric constraints between groups of segments can be used to prune scene interpretation hypotheses and to generate the most plausible interpretation.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/3228234ab663758d7439d9ee8f30c8fb29db8e7f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Indoor Scene Understanding with RGB-D Images: Bottom-up Segmentation, Object Detection and Semantic Segmentation (2015, background): “Thus, [21] recovers walk-able surfaces by reasoning on the location and shape of furniture, [34, 35] reason about the 3D geometry of the room and objects, while [19] focuses on interpreting the scene in a human-centric perspective.”
Understanding Indoor Scenes Using 3D Geometric Phrases (2013, background): “A layout estimator [12, 10, 16, 27] will tell you, with different uncertainty, how to fit a box to the room.”
Pop-up SLAM: Semantic monocular plane SLAM for low-texture environments (2016, methodology): “Two representative examples are cuboidal room box model proposed based on vanishing point by Hedau et al. [5] and fixed building model collections based on line segments by Lee et al. [6].”
Single Image 3D without a Single 3D Image (2015, methodology): “Since our label space is Manhattan world, we removed images where vanishing points could not be estimated, identified as ones where the independent estimates of [28, 20] disagree.”
Semantic structure from motion with points, regions, and objects (2012, background): “Among these works (e.g. [12] [15] [11] [25] [10] [20] [6] [13]), most assume that only a single image is available, whereas a handful of them model this problem with multiple images.”
Bayesian geometric modeling of indoor scenes (2012, unknown): “We are motivated by the recent advancements in the task of recovering the 3D geometry of indoor scenes [2, 3, 5, 9, 10, 12, 15, 16].”
Scene Understanding by Reasoning Stability and Safety (2015, background): “…e.g., masses, gravity potentials, contact area, etc. (2) Physical reasoning The vision communities have studied the physical properties based on a single image for the “block world” in the past three decades (Biederman et al. 1982; Gupta et al. 2010, 2011; Zhao and Zhu 2011; Lee et al. 2009, 2010).”
Overview of Environment Perception for Intelligent Vehicles (2017, background): “, 2009 [185] indoor 3D cuboid from a single image • Dataset: 54 images of indoor scenes • Results:”
Support Surface Prediction in Indoor Scenes (2013, background): “One line of work is to predict boxy layouts from RGB images [6, 7, 10, 8, 5, 12, 11], in which occlusion and lack of 3D sensors is combatted with simple scene models and strong priors.”
Three-Dimensional Object Detection and Layout Prediction Using Clouds of Oriented Gradients (2016, methodology): “Building on [22] and [15], Hedau et al.”
Efficient Structured Prediction with Latent Variables for General Graphical Models (2012, methodology): “Following existing approaches (Hedau et al., 2009; Lee et al., 2010), we employ F = 55 features based on geometric context (GC) and orientation maps (OM) and refer the interested reader to (Hoiem et al., 2007) and (Lee et al., 2009) for respective details.”
Predicting Failures of Vision Systems (2014, methodology): “BASESYS involves detecting lines in image x i and clustering them in 3 directions (and outliers) [33] to estimate the VPs.”
Scene Parsing by Integrating Function, Geometry and Appearance Models (2013, methodology): “Overview of our approach: On top of a series of recent studies of computing the 3D bounding boxes of indoor objects and the room layout [9, 12, 13, 21, 17, 16, 10, 26, 4, 3, 19], our model is developed based on the following observations of the function-geometry-appearance (FGA) hierarchy as shown i”
Beyond Point Clouds: Scene Understanding by Reasoning Geometry and Physics (2013, background): “The vision communities have studied the physical properties based on single image for the ”block world” in the past three decades [3, 8, 9, 21, 15, 14]).”
IM2CAD (2016, methodology): “When combined with a box-ﬁtting step of [15, 22], we achieve competitive result of 10 .”
Blending Learning and Inference in Conditional Random Fields (2016, methodology): “For image x and a given layout hypothesis y = (y1, . . . , y4) we compute a 55-dimensional feature vector φ(x, y) which is constructed based on geometric context (Hoiem et al., 2007) and orientation maps (Lee et al., 2009) illustrated in the middle and bottom row of Figure 3 respectively.”
Unfolding an Indoor Origami World (2014, unknown): “developing constrained models for the prediction of room layout [10] as well as features [6,23,27] and effective methods for inference [4,22,31,32].”
Estimating Spatial Layout of Rooms using Volumetric Reasoning about Objects and Surfaces (2010, methodology): “We use two sources of surface geometry, orientation map [11] and geometric context [8], which serve as features in the cost function.”