Cross-View Image Matching for Geo-Localization in Urban Environments

By Yicong Tian, Mubarak Shah. 2017

225 citations recorded by the source corpus.

In this paper, we address the problem of cross-view image geo-localization. Specifically, we aim to estimate the GPS location of a query street view image by finding the matching images in a reference database of geo-tagged birds eye view images, or vice versa. To this end, we present a new framework for cross-view image geo-localization by taking advantage of the tremendous success of deep convolutional neural networks (CNNs) in image classification and object detection. First, we employ the Faster R-CNN to detect buildings in the query and reference images. Next, for each building in the query image, we retrieve the k nearest neighbors from the reference buildings using a Siamese network trained on both positive matching image pairs and negative pairs. To find the correct NN for each query building, we develop an efficient multiple nearest neighbors matching method based on dominant sets. We evaluate the proposed framework on a new dataset that consists of pairs of street view and birds eye view images. Experimental results show that the proposed method achieves better geo-localization accuracy than other approaches and is able to generalize to images at unseen locations.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

1. Yi Yang
2. Zhedong Zheng
3. Liu Liu
4. Hongdong Li
5. Ke Li
6. Junwei Han
7. Yunchao Wei
8. Gong Cheng

In Computer Vision and Pattern Recognition

1. Yi Yang
2. Luc Van Gool
3. Junwei Han
4. Dacheng Tao
5. Gong Cheng
6. Yunchao Wei
7. Ming-Hsuan Yang
8. Xiaogang Wang

Citations in the corpus

• CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps (2018, background): “Cross-view geolocation recognition makes additional use of satellite or aerial imagery to determine query locations [37,38,39,40].”
• CVM-Net: Cross-View Matching Network for Image-Based Ground-to-Aerial Geo-Localization (2018, background): “As a result, matching ground view photos to aerial imagery gradually becomes an increasingly popular geo-localization approach [5, 22, 37, 23, 49, 50, 46, 41, 53, 43].”
• Lending Orientation to Neural Networks for Cross-View Geo-Localization (2019, background): “Paper [17, 29] tackled the ground to bird-eye view (45-degree oblique views) matching problem, which is a relatively easier task in the sense that more overlaps (e.”
• Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark (2019, background): “…tasks (Chen et al., 2018; Cheng et al., 2018a; Clément et al., 2013; Ding et al., 2017; Hinton et al., 2012; Hou et al., 2017; Krizhevsky et al., 2012; Mikolov et al., 2012; Tian et al., 2017; Tompson et al., 2014; Wei et al., 2018) because of their powerful feature representation capabilities.”
• Collaborative Multi-Robot Search and Rescue: Planning, Coordination, Perception, and Active Vision (2020, background): “sensors can be achieved with image matching [110], or lidar data matching [111].”
• University-1652: A Multi-view Multi-source Benchmark for Drone-based Geo-localization (2020, methodology): “[31] collect image pairs for urban localization.”
• Cross-View Policy Learning for Street Navigation (2019, background): “proposed to learn a joint embedding space using a deep neural network between street views and aerial views [25, 41].”