Deep Image Homography Estimation

By Daniel DeTone, Tomasz Malisiewicz, Andrew Rabinovich. arXiv.org, 2016

518 citations recorded by the source corpus.

We present a deep convolutional neural network for estimating the relative homography between a pair of images. Our feed-forward network has 10 layers, takes two stacked grayscale images as input, and produces an 8 degree of freedom homography which can be used to map the pixels from the first image to the second. We present two convolutional neural network architectures for HomographyNet: a regression network which directly estimates the real-valued homography parameters, and a classification network which produces a distribution over quantized homographies. We use a 4-point homography parameterization which maps the four corners from one image into the second image. Our networks are trained in an end-to-end fashion using warped MS-COCO images. Our approach works without the need for separate local feature detection and transformation estimation stages. Our deep models are compared to a traditional homography estimator based on ORB features and we highlight the scenarios where HomographyNet outperforms the traditional technique. We also describe a variety of applications powered by deep homography estimation, thus showcasing the flexibility of a deep learning approach.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/e353bd01a250983d4ed10ed990561246eab8d932

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Global Pose Estimation with an Attention-Based Recurrent Network (2018, background): “They have been successfully applied to several related multiview vision problems, including learning optical flow [11], depth [28], homography between frame pairs [9], and localization [5] and re-localization problems.”
Image Matching from Handcrafted to Deep Features: A Survey (2020, methodology): “Similar methods regarding deep networks as a regressor can directly learn the parameter transform model from image pairs, such as Fundamental (Poursaeed et al. 2018), Homography (DeTone et al. 2016), and non-rigid deformation (Rocco et al. 2017).”
DSAC — Differentiable RANSAC for Camera Localization (2016, background): “Recently, deep learning has been shown to be highly successful at image recognition tasks [37, 17, 13, 31], and, increasingly, in other domains including geometry [10, 19, 20, 9].”
Unsupervised label noise modeling and loss correction (2019, background): “Convolutional Neural Networks (CNNs) have recently become the par excellence base approach to deal with many computer vision tasks (DeTone et al., 2016; Ono et al., 2018; Beluch et al., 2018; Redmon et al., 2016; Zhao et al., 2017; Krishna et al., 2017).”
SuperPoint: Self-Supervised Interest Point Detection and Description (2017, background): “Also, Geometric Matching Networks [20] and Deep Image Homography Estimation [4] use a similar self-supervision strategy to create training data for estimating global transformations.”
Taskonomy: Disentangling Task Transfer Learning (2018, background): “Room Layout Estimation Estimating and aligning a 3D bounding box is a mid-level task that includes vanishing point estimation as a sub-problem, and has applications for robotic navigation [38], scene reconstruction [15], and augmented reality [8].”
VINet: Visual-Inertial Odometry as a Sequence-to-Sequence Learning Problem (2017, methodology): “We now describe the detailed structure of our VINet model which integrates the following components.”
Visual SLAM and Structure from Motion in Dynamic Environments (2018, methodology): “Instead of estimating general motion similar to fundamental matrix, DeTone et al. [32] proposed “HomographyNet” to train a CNN for computing homography between two frames using 4-point parameterization of homography.”
Unsupervised Deep Homography: A Fast and Robust Homography Estimation Model (2017, methodology): “We evaluate these methods on a synthetic dataset similar to the dataset used in [24], and on a real-world aerial image dataset.”
Convolutional Neural Network Architecture for Geometric Matching (2017, methodology): “Re-placing our correlation-based matching layer with feature concatenation or subtraction, as proposed in [14] and [30], respectively, incurs a large performance drop.”
SuperGlue: Learning Feature Matching With Graph Neural Networks (2019, methodology): “Dataset: We generate image pairs by sampling random homographies and applying random photometric distortions to real images, following a recipe similar to [14, 16, 42, 41].”