Object Recognition by Scene Alignment

By Bryan C. Russell, Antonio Torralba, Ce Liu, Rob Fergus, William T. Freeman. 2007

132 citations recorded by the source corpus.

Current object recognition systems can only recognize a limited number of object categories; scaling up to many categories is the next challenge. We seek to build a system to recognize and localize many different object categories in complex scenes. We achieve this through a simple approach: by matching the input image, in an appropriate representation, to images in a large training set of labeled images. Due to regularities in object identities across similar scenes, the retrieved matches provide hypotheses for object identities and locations. We build a probabilistic model to transfer the labels from the retrieval set to the input image. We demonstrate the effectiveness of this approach and study algorithm component contributions using held-out test sets from the LabelMe database.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Ensemble of exemplar-SVMs for object detection and beyond (2011, background): “Because most detections show good alignment to their associated exemplar, it is possible to transfer any available exemplar meta-data (segmentation, geometric structure, 3D model, etc.) directly onto the detections, which can then be used as part of overall scene understanding.”
An empirical study of context in object detection (2009, background): “2D scene gist uses global statistics of an image to capture the “gist” of the visual experience [28, 32].”
Nonparametric Scene Parsing via Label Transfer (2011, methodology): “For comparison, the groundtruth user annotation of the query is displayed in (Fig.”
SIFT Flow: Dense Correspondence across Different Scenes (2008, background): “Inspired by the recent progress in large image database methods [11,12,13], and the traditional optical flow estimation for temporally adjacent (and thus visually similar) frames, we create a large database so that for each query image we can retrieve a set of visually similar scenes.”
Spatial pyramid matching (2009, unknown): “Today, local features continue to enjoy a great degree of success, but at the same there is a notable resurgence of interest in global appearance-based methods (Oliva and Torralba, 2001; Hays and Efros, 2007; Russell et al., 2007; Torralba et al., 2007, 2008).”
Superparsing (2010, methodology): “SIFT Flow dataset [15] Barcelona dataset [7] Semantic Geometric Semantic Geometric Baseline 74.75 [15 Our ﬁnal system on the SIFT Flow dataset achieves a classiﬁcation rate of 76.9%.”
Scene parsing with Multiscale Feature Learning, Purity Trees, and Optimal Covers (2012, methodology): “[24], is derived from the LabelMe subset used in [21].”
Beyond pixels: exploring new representations and applications for motion analysis (2009, methodology): “I also participated in object recognition by scene alignment [90] with Dr.”
Object localization in ImageNet by looking out of the window (2015, background): “Since then, many object detectors [2, 17, 25, 28, 29, 32] have employed such global context to re-score their detections, thereby removing out-of-context false-positives.”
Building a database of 3D scenes from user annotations (2009, background): “Techniques for aligning images [10, 25, 20] may also benefit from such data.”
Ieee Transactions on Pattern Analysis and Machine Intelligence 1 80 Million Tiny Images: a Large Dataset for Non-parametric Object and Scene Recognition (unknown): “They have also been used for global image matching (e.g. estimation of human pose [36]), character recognition [4], and object recognition [5], [34].”
Measuring and Predicting Object Importance (2011, background): “edu come an active area of research (Everingham et al. 2008; Russell et al. 2007).”
LabelMe: Online Image Annotation and Applications (2010, methodology): “Techniques for aligning images [20], [26], [50] may also benefit from such data.”
Actions in context (2009, methodology): “For example, scene context is used to impose spatial priors on the location of objects in the image [21, 26].”
Modeling Scene and Object Contexts for Human Action Retrieval With Few Examples (2011, background): “Wu et al. [19] investigated contexts from radio-frequency identification-tagged objects for kitchen activity recognition.”
Recognition by association via learning per-exemplar distances (2008, background): “On the other hand, global approaches [20, 14] utilize information over the entire image (the gist of the scene) to provide guidance for many separate object detectors.”