Learning Two-View Correspondences and Geometry Using Order-Aware Network

By Jiahui Zhang, Dawei Sun, Zixin Luo, Anbang Yao, Lei Zhou, Tianwei Shen, Yurong Chen, Long Quan, Hongen Liao. IEEE International Conference on Computer Vision, 2019

368 citations recorded by the source corpus.

Establishing correspondences between two images requires both local and global spatial context. Given putative correspondences of feature points in two views, in this paper, we propose Order-Aware Network, which infers the probabilities of correspondences being inliers and regresses the relative pose encoded by the essential matrix. Specifically, this proposed network is built hierarchically and comprises three novel operations. First, to capture the local context of sparse correspondences, the network clusters unordered input correspondences by learning a soft assignment matrix. These clusters are in a canonical order and invariant to input permutations. Next, the clusters are spatially correlated to form the global context of correspondences. After that, the context-encoded clusters are recovered back to the original size through a proposed upsampling operator. We intensively experiment on both outdoor and indoor datasets. The accuracy of the two-view geometry and correspondences are significantly improved over the state-of-the-arts.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/d46da5ef2fbe6602bcc890e6207a8c8da7933e58

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformer (2022, methodology): “Differing from detector-based matching methods, which typical involve detecting [11,9,10,8], describing [26,27,28,12,29] and matching [30,31,32,33,34,35,36] a set of keypoints, detector-free matching consumes a pair of images and output correspondences in one shot.”
ACNe: Attentive Context Normalization for Robust Permutation-Equivariant Learning (2020, methodology): “In OANet [58], order-aware filtering was utilized to incorporate context and spatial correlation.”
Image Matching from Handcrafted to Deep Features: A Survey (2020, methodology): “Zhang et al. (2019b) reported that some sequences in this dataset do not provide camera poses.”
DISK: Learning local features with policy gradient (2020, methodology): “Note that we only compare with submissions using the built-in feature matcher, based on the l2 distance between descriptors, instead of neural-network based matchers [46, 49, 33], which combined with state-of-the-art features obtain the best overall results.”
A review of multimodal image matching: Methods and applications (2021, background): “[345] focused more on geometrical recovery in their order-aware networks.”
Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition (2021, methodology): “One avenue for future work is the following: while we match Patch-NetVLAD features using mutual nearest neighbors with subsequent spatial verification using RANSAC, recent deep learned matchers [62, 88] could further improve the global re-localization performance of the algorithm.”
LightGlue: Local Feature Matching at Light Speed (2023, methodology): “Those are filtered out by heuristics, like Lowe’s ratio test [41] or the mutual check, inlier classifiers [44, 82], and by robustly fitting geometric models [22, 7].”
ASLFeat: Learning Local Features of Accurate Shape and Localization (2020, background): “Designing powerful local features is an essential basis for a broad range of computer vision tasks [33, 47, 48, 32, 44, 17, 44].”