Recognizing scene viewpoint using panoramic place representation

By Jianxiong Xiao, Krista A. Ehinger, Aude Oliva, Antonio Torralba. 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012

401 citations recorded by the source corpus.

We introduce the problem of scene viewpoint recognition, the goal of which is to classify the type of place shown in a photo, and also recognize the observer's viewpoint within that category of place. We construct a database of 360° panoramic images organized into 26 place categories. For each category, our algorithm automatically aligns the panoramas to build a full-view representation of the surrounding place. We also study the symmetry properties and canonical viewpoint of each place category. At test time, given a photo of a scene, the model can recognize the place category, produce a compass-like indication of the observer's most likely viewpoint within that place, and use this information to extrapolate beyond the available view, filling in the probable visual layout that would appear beyond the boundary of the photo.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/38c17d7f444f93922c84e99f778dfafa3f200772

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Unsupervised Non-parametric Geospatial Modeling from Ground Imagery (2014, background): “beaches or theaters) and registration of the views into the same frame of reference [9].”
Learning to Look Around: Intelligently Exploring Unseen Environments for Unknown Tasks (2017, methodology): “For both SUN360 active scene recognition and ModelNet-10 active object recognition, our unsupervised policies perform on par with the end-to-end active categorization policy of [29], easily outperforming random-policy, 1-view, and large-action.”
SUN Database: Exploring a Large Collection of Scene Categories (2014, methodology): “We refer to this dataset as the SUN (Scene UNderstanding) database1 Xiao et al. (2010).”
Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling (2019, background): “lls, ﬂoor, and ceiling. Existing public datasets (e.g., PanoContext [31] and LayoutNet [34]) assume a simple cuboid-shape layout. PanoContext [31] collects about 500 panoramas from the SUN360 dataset [26], LayoutNet [34] extends the layout annotations to include panoramas from 2D-3D-S [2]. Recently,”
Pano2Vid: Automatic Cinematography for Watching 360° Videos (2016, background): “5◦ (corresponding to a typical 28mm focal length fullframe Single Lens Reflex Camera [39]) with a 4:3 aspect ratio.”
SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels (2013, background): “[23] proposed a “place-centric” representation for scene understanding, but they only focus on 2D recognition and the place-centric representation built in [23] is not a complete 3D representation of the full extent of the space.”
Flat2Sphere: Learning Spherical Convolution for Fast Features from 360° Imagery (2017, methodology): “The SUN360 project samples multiple perspective images to perform scene viewpoint recognition [31].”
Matterport3D: Learning from RGB-D Data in Indoor Environments (2017, background): “Previous panorama datasets have provided either no depths at all [42] or approximate depths synthesized from meshes [1].”
Learning Spherical Convolution for Fast Features from 360° Imagery (2017, methodology): “5° to W=640 pixels following SUN360 [38].”
DeepSphere: Efficient spherical Convolutional Neural Network with HEALPix sampling for cosmological applications (2018, methodology): “This approach has been extensively used for cosmological maps [9, 11, 12, 23] and omnidirectional imaging [48, 49].”
Emergence of exploratory look-around behaviors through active observation completion (2019, methodology): “ucibility, we evaluate our approach on two widely used datasets: SUN360 Dataset for Scenes For this dataset, our limited ﬁeld-of-view (60) agent attempts to complete an omnidirectional scene. SUN360 (32) has spherical panoramas of 26 diverse categories. The dataset consists of 6,174 training, 1,013 ”
Sidekick Policy Learning for Active Visual Exploration (2018, methodology): “– SUN360: SUN360 [66] consists of high resolution spherical panoramas from multiple scene categories.”
Manipulating Attributes of Natural Scenes via Hallucination (2018, background): “e is very wide, like a panorama, center-cropping omits most of the source image. We analyze how this affects the overall performance our framework on a couple of panoramic images from SUN360 dataset [Xiao et al. 2012]. We present attribute manipulation results for one of these images in Fig. 14 and ”
Learning to predict indoor illumination from a single image (2017, methodology): “First, we rely on a database of LDR panoramas [Xiao et al. 2012].”
Deep reverse tone mapping (2017, methodology): “The input test images are obtained from Places205 dataset [Zhou et al. 2014] and SUN360 dataset [Xiao et al. 2012].”
Learning to Recover 3D Scene Shape from a Single Image (2020, methodology): “The model of [18] is trained on the in-the-wild SUN360 [45] dataset.”