Scene Summarization for Online Image Collections

By Ian Simon, Noah Snavely, S. M. Seitz. IEEE International Conference on Computer Vision, 2007

406 citations recorded by the source corpus.

We formulate the problem of scene summarization as selecting a set of images that efficiently represents the visual content of a given scene. The ideal summary presents the most interesting and important aspects of the scene with minimal redundancy. We propose a solution to this problem using multi-user image collections from the Internet. Our solution examines the distribution of images in the collection to select a set of canonical views to form the scene summary, using clustering techniques on visual features. The summaries we compute also lend themselves naturally to the browsing of image collections, and can be augmented by analyzing user-specified image tag data. We demonstrate the approach using a collection of images of the city of Rome, showing the ability to automatically decompose the images into separate scenes, and identify canonical views for each scene.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/65cf5bdb2a29e30bc35e085ca27189cac6994b97

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Finding iconic images (2009, background): “[20, 15] which relies on geometric constraints to find representative images of specific instances of rigid objects (building facades) instead of general object categories.”
Video summarization by learning submodular mixtures of objectives (2015, methodology): “for document [23, 21, 22] and also image collection [31, 36] and video summarization using keyframes [12].”
Large scale mining and retrieval of visual data in a multimodal context (2008, methodology): “The work most similar to ours is probably [Simon et al., 2007; Snavely et al., 2006].”
Computing iconic summaries of general visual concepts (2008, methodology): “[23] have proposed a method for summarizing landmark image collections by directly exploiting 3D scene structure.”
Image webs: Computing and exploiting connectivity in image collections (2010, background): “In [21], clustering is performed in the image graph to choose a set of representative images for an image collection.”
World-scale mining of objects and events from community photo collections (2008, methodology): “The work most similar to ours is probably [25, 29].”
Building Rome in a day (2009, background): “While reconstructions generated from such collections do not capture a complete covering of scene surfaces, the coverage improves over time, and can be complemented by adding aerial or street-side images.”
Image Clustering Using Local Discriminant Models and Global Integration (2010, methodology): “In [16] and [28], the image clustering algorithms were utilized to extract a small subset of representative images for representing the diverse views of the landmarks.”
Generating diverse and representative image search results for landmarks (2008, background): “Given the reduced set of images, our approach for generating a diverse and representative set of images for a landmark is based on identifying “canonical views” [20, 18].”
Non-monotone Submodular Maximization with Nearly Optimal Adaptivity and Query Complexity (2018, methodology): “in applications of machine learning such as data summarization (Simon et al., 2007; Sipos et al., 2012; Tschiatschek et al., 2014), feature selection (Das & Kempe, 2008; Khanna et al.”
Visual landmark recognition from Internet photo collections: A large-scale evaluation (2014, methodology): “ter. Crandall et al. [9] deal with uninformative tags in a more general way by dividing the number of occurrences of a tag in a cluster by its total number of occurrences in the dataset. Simon et al. [10] additionally account for tags that are only used by individual users by computing a conditional”
I know what you did last summer: object-level auto-annotation of holiday snaps (2009, methodology): “In this paper, we combine a data collection process similar to [9] [18] [20] [22] with object retrieval methods [7] [6] [10] [15] [16] [17] [21] [8] to obtain a large-scale, retrieval-based auto annotation system.”
Photo2Trip: generating travel routes from geo-tagged photos for trip planning (2010, background): “[10] mined city landmarks from blogs by exploiting graphic models, while [8], [13], [11] and [5] attempted to visualize, recognize, describe and summarize a scene or a landmark by leveraging geo-tagged photos.”
Retrieving landmark and non-landmark images from community photo collections (2010, methodology): “Matching features are linked into connected components in [19], [30], and we need a similar compact representation, that is, more compact than storing features of individual views.”
IM2GPS: estimating geographic information from a single image (2008, methodology): “Similar feature-based geometric matching approaches have also been successfully applied to co-registering online photographs of famous landmarks for browsing [14] and summarization [13],”
Clues from the beaten path: Location estimation with bursty sequences of tourist photos (2011, background): “Access to many users’ online collections together with such effective local feature matching methods have led to compelling new ways to browse community photo collections [18, 17], and to discover iconic sites [16, 13, 8, 4].”
Why do deep convolutional networks generalize so poorly to small image transformations? (2018, result): “Consistent with previous results on "dataset bias" [37, 33, 2, 44, 45, 32] we find that the ImageNet dataset is not invariant to translations and rescalings.”
Mapping the world's photos (2009, methodology): “As in [22], we pose canonical image selection as a graph problem.”