Geometric context from a single image

By Derek Hoiem, Alexei A. Efros, Martial Hebert. 2005

799 citations recorded by the source corpus.

Many computer vision algorithms limit their performance by ignoring the underlying 3D geometric structure in the image. We show that we can estimate the coarse geometric properties of a scene by learning appearance-based models of geometric classes, even in cluttered natural scenes. Geometric classes describe the 3D orientation of an image region with respect to the camera. We provide a multiple-hypothesis framework for robustly estimating scene structure from a single image and obtaining confidences for each geometric label. These confidences can then be used to improve the performance of many other applications. We provide a thorough quantitative evaluation of our algorithm on a set of outdoor images and demonstrate its usefulness in two applications: object detection and automatic single-view reconstruction.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Zero-Shot Learning Through Cross-Modal Transfer (2013, background): “This is usually achieved by either sharing of feature representations [2], model parameters [12] or via similar context [14].”
SUN database: Large-scale scene recognition from abbey to zoo (2010, background): “Previous attempts to characterize environmental scenes have capitalized on uncovering a manageable set of dimensions, features or objects that are correlated with the semantic of purpose of a space [21, 17, 7, 32, 12].”
Designing deep networks for surface normal estimation (2014, methodology): “Recent work has focused on using edges [25], super pixels [28] or segments [15] as primitives for reasoning.”
Synthetic Data for Text Localisation in Natural Images (2016, background): “[15] categorise image regions into ground-plane, vertical plane or sky from a single image and use it to generate “pop-ups” by decomposing the image into planes [14].”
Ensemble of exemplar-SVMs for object detection and beyond (2011, methodology): “We mine hard negatives from out-ofclass images in the train set and perform calibration using all positive and negative images in trainval (See Section 3.1).”
3D Shape Induction from 2D Views of Multiple Objects (2016, methodology): “Recognition-based methods have been used to estimate geometry of out-door scenes [14, 26], indoor environments [9, 27], and objects [1, 25].”
IM2GPS: estimating geographic information from a single image (2008, methodology): “Geometric Context: Finally, we compute the geometric class probabilities for image regions using the method of Hoiem et al. [5].”
Using Multiple Segmentations to Discover Objects and their Extent in Image Collections (2006, background): “[13] have proposed a surprisingly effective way of utilizing image segmentation without suffering from its shortcomings.”
Computational Baby Learning (2014, background): “ot learning [10] which learns visual object classiﬁers by using very few samples. Most of the one-shot learning methods are based on the feature representation transfer [2], similar geometric context [15] or cross-modal knowledge transfer [26]. However, their performance is far from that of the stat”
An empirical study of context in object detection (2009, methodology): “For class appearance, we compute histograms of K-means clustered L*a*b* color (K=128) and textons [38] (K=256), as well as geometric context confidences [16] and probability of background confidences (trained using the geometric context algorithm on examples from LabelMe [30]), quantized to ten valu”
Putting Objects in Perspective (2006, background): “[18]), 3D scene geometry estimation [11], and camera viewpoint estimation.”
Robust Higher Order Potentials for Enforcing Label Consistency (2008, methodology): “They have been shown to give decent results which have proved to be useful for many applications [10, 11, 32].”
Dataset Issues in Object Recognition (2006, background): “7 New algorithms that exploit context for object recognition [16,28] require databases with many labeled object classes embedded in complex scenes.”
3-D Depth Reconstruction from a Single Still Image (2007, background): “In work that is contemporary to ours, Hoiem, Efros and Herbert [26; 27] built a simple “pop-up” type 3-d model from an image by classifying the image into ground, vertical and sky.”
Indoor Segmentation and Support Inference from RGBD Images (2012, methodology): “Our overall approach of incorporating geometric priors to improve scene interpretation is most related to a set of image-based single-view methods (e.g. [1–7]).”
Mesh R-CNN (2019, methodology): “Some methods predict the orientation [10, 20] or 3D pose [31, 44, 66] of known shapes.”
Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scene (2017, background): “In the learning-based era, this has mainly taken the form of estimating view-based per-pixel 3D properties of scenes such as depth [7, 26] or orientation information [9, 16].”
Salient Object Detection: A Discriminative Regional Feature Integration Approach (2013, methodology): “The two features are extracted independently from each region like the feature extraction algorithm in image labeling [19].”