Learning to Match Aerial Images with Deep Attentive Architectures

By Hani Altwaijry, Eduard Trulls, James Hays, Pascal Fua, Serge J. Belongie. 2016

75 citations recorded by the source corpus.

Image matching is a fundamental problem in Computer Vision. In the context of feature-based matching, SIFT and its variants have long excelled in a wide array of applications. However, for ultra-wide baselines, as in the case of aerial images captured under large camera rotations, the appearance variation goes beyond the reach of SIFT and RANSAC. In this paper we propose a data-driven, deep learning-based approach that sidesteps local correspondence by framing the problem as a classification task. Furthermore, we demonstrate that local correspondences can still be useful. To do so we incorporate an attention mechanism to produce a set of probable matches, which allows us to further increase performance. We train our models on a dataset of urban aerial imagery consisting of 'same' and 'different' pairs, collected for this purpose, and characterize the problem via a human study with annotations from Amazon Mechanical Turk. We demonstrate that our models outperform the state-of-the-art on ultra-wide baseline matching and approach human accuracy.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Geolocation Estimation of Photos Using a Hierarchical Model and Scene Classification (2018, background): “In this context, solutions are presented that match an aerial query image against a reference dataset containing satellite images in a wide baseline approach [2,6,43].”
Convolutional Neural Network Architecture for Geometric Matching (2017, methodology): “Extracted descriptors are then compared with an appropriate distance measure [8, 29, 47], by directly outputting a similarity score [24, 54], or even by directly outputting a binary matching/non-matching decision [3].”
Unsupervised Deep Learning for Optical Flow Estimation (2017, methodology): “(Altwaijry et al. 2016) uses STN for image similarity verification as the model can be trained with standard backpropagation, unlike the attention mechanisms of (Mnih et al.”
Localizing and Orienting Street Views Using Overhead Imagery (2016, methodology): “Most recently, Altwaijry et al. [2] use a deep attentive architecture to match aerial images across wide baselines.”