What makes Paris look like Paris?

By Carl Doersch, Saurabh Singh, Abhinav Kumar Gupta, Josef Sivic, Alexei A. Efros. 2012

846 citations recorded by the source corpus.

Given a large repository of geotagged imagery, we seek to automatically find visual elements, e. g. windows, balconies, and street signs, that are most distinctive for a certain geo-spatial area, for example the city of Paris. This is a tremendously difficult task as the visual features distinguishing architectural elements of different places can be very subtle. In addition, we face a hard search problem: given all possible patches in all images, which of them are both frequently occurring and geographically informative? To address these issues, we propose to use a discriminative clustering approach able to take into account the weak geographic supervision. We show that geographically representative image elements can be discovered automatically from Google Street View imagery in a discriminative manner. We demonstrate that these elements are visually interpretable and perceptually geo-informative. The discovered visual elements can also support a variety of computational geography tasks, such as mapping architectural correspondences and influences within and across cities, finding representative elements at different geo-spatial scales, and geographically-informed image retrieval.

Subfields: Computer Vision and Pattern Recognition, Media Technology, Ocean Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

In Ocean Engineering

Citations in the corpus

Context Encoders: Feature Learning by Inpainting (2016, methodology): “We experiment with images from two datasets: Paris StreetView [8] and ImageNet [37] without using any of the accompanying labels.”
Unsupervised Visual Representation Learning by Context Prediction (2015, background): “Finally, we connect conv6b to a fully connected layer—called fc7—which in turn connects to a final fc8 layer that produces the outputs that feed into the softmax.”
EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning (2019, methodology): “Note that we did not include values for PConv over the CelebA and Paris StreetView datasets as the source code was not available at the time of writing.”
Image-to-Image Translation with Conditional Adversarial Networks (2016, methodology): “to models on humandrawn sketches from [19]. Day!night, trained on [33]. Thermal!color photos, trained on data from [27]. Photo with missing pixels!inpainted photo, trained on Paris StreetView from [14]. Details of training on each of these datasets are provided in the supplemental materials online. ”
Scaling and Benchmarking Self-Supervised Visual Representation Learning (2019, background): “One form of discriminative learning combines clustering with hand-crafted features to learn visual representations such as image-patches [13, 62], object discovery [57, 63].”
CrossTransformers: spatially-aware few-shot transfer (2020, background): “Discriminative parts [5, 10, 22, 34, 42, 83] and visual words [74, 100] have a rich history, and have found applications in deformable-parts models [26], and classification [70, 100].”
Generating Visual Explanations (2016, background): “A variety of computer vision methods have focused on discovering visual features which can help “explain” an image classiﬁcation decision [20,21,22].”
Image-Based Recommendations on Styles and Substitutes (2015, background): “There has been some interest of late in modeling the visual style of places [6, 27], and objects [39].”
Learning to Explain With Complemental Examples (2018, background): “Prototype selection [25, 7, 13, 16, 25, 10] or machine teaching [18] can be considered as example-based explanations.”
Multimodal Explanations: Justifying Decisions and Pointing to the Evidence (2018, methodology): “Some methods find discriminative visual patches [7, 11] whereas others aim to understand what specific neurons represent [12, 38, 39].”
Unsupervised Deep Embedding for Clustering Analysis (2015, methodology): “re 10 classes with 1300 examples each. It also contains 100000 unlabeled images of the same resolution (Coates et al., 2011). We also used the unlabeled set when training our autoencoders. Similar to Doersch et al. (2012), we concatenated HOG feature and a 8-by-8 color map to use as input to all alg”
Context as Supervisory Signal: Discovering Objects with Predictable Context (2014, methodology): “At a high level, our pipeline is similar to algorithms for mid-level patch discovery [16,15], especially in the early stages.”
Large-Scale Weakly-Supervised Pre-Training for Video Action Recognition (2019, background): “Learning from weak supervision: Given the known challenges in gathering exhaustive annotations for various image and video tasks, leveraging object labels and other meta information to supply weak supervision [7, 15, 19, 22, 30, 31, 39, 40, 50, 52, 53, 54, 55, 56, 59] is a widely-adopted approach.”
High-Resolution Image Inpainting Using Multi-scale Neural Patch Synthesis (2016, methodology): “The Paris StreetView contains 14,900 training images and 100 test images.”
CliqueCNN: Deep Unsupervised Exemplar Learning (2016, methodology): “The exemplar paradigm has been successfully employed in diverse areas such as segmentation [11], grouping [10], instance retrieval [2, 19], and object recognition [15, 5].”
Learning Visual Question Answering by Bootstrapping Hard Attention (2018, background): “One line of work on discriminative patches builds a representation by selecting some patches and ignoring others, which has proved useful for object detection and classiﬁ-cation [56,57,58], and especially visualization [59].”