SOSNet: Second Order Similarity Regularization for Local Descriptor Learning

By Yurun Tian, Xin-Yao Yu, Bin Fan, Fuchao Wu, Huub Heijnen, Vassileios Balntas. Computer Vision and Pattern Recognition, 2019

385 citations recorded by the source corpus.

Despite the fact that Second Order Similarity (SOS) has been used with significant success in tasks such as graph matching and clustering, it has not been exploited for learning local descriptors. In this work, we explore the potential of \sos in the field of descriptor learning by building upon the intuition that a positive pair of matching points should exhibit similar distances with respect to other points in the embedding space. Thus, we propose a novel regularization term, named Second Order Similarity Regularization (SOSR), that follows this principle. By incorporating SOSR into training, our learned descriptor achieves state-of-the-art performance on several challenging benchmarks containing distinct tasks ranging from local patch retrieval to structure from motion. Furthermore, by designing a von Mises-Fischer distribution based evaluation method, we link the utilization of the descriptor space to the matching performance, thus demonstrating the effectiveness of our proposed SOSR. Extensive experimental results, empirical evidence， and in-depth analysis are provided, indicating that SOSR can significantly boost the matching performance of the learned descriptor.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/150233ece66acabaf5ab7c555270e2cc4104791a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

COTR: Correspondence Transformer for Matching Across Images (2021, background): “Popular patch descriptors range from hand-crafted [34, 9] to learned [42, 66, 17] ones.”
Image Matching from Handcrafted to Deep Features: A Survey (2020, background): “2017; Weinberger and Saul 2009) and descriptor learning (Simo-Serra et al. 2015; Balntas et al. 2016a, 2017; Zhang et al. 2017c; Mishchuk et al. 2017; Wei et al. 2018; He et al. 2018; Tian et al. 2019; Luo et al. 2019), according to the output of deep learning-based descriptors.”
Image Matching Across Wide Baselines: From Paper to Practice (2020, methodology): “2018), SOSNet (Tian et al. 2019), ContextDesc (Luo et al.”
Grounding Image Matching in 3D with MASt3R (2024, methodology): “Departing from the former handcrafted methods like SIFT [52,71], modern approaches have been shifting towards learning-based data-driven schemes for detecting keypoints [8,60,97,117], describing them [7,33, 37,88] or both at the same time [10,21,53,54,70,98].”
DISK: Learning local features with policy gradient (2020, methodology): “The first wave of local features involving deep networks featured descriptors learned from patches extracted on SIFT keypoints [48, 14, 38] and some of their successors, such as HardNet [25], SOSNet [40], and LogPolarDesc [13], are still state-of-the-art.”
ASLFeat: Learning Local Features of Accurate Shape and Localization (2020, neutral): “Most existing descriptor learning methods [19, 18, 21, 37, 36, 40] do not explicitly model the local shape, but rely on geometric data augmentation (scaling/rotational perturbation) or hand-crafted shape estimation (scale/rotation estimation [17, 29]) to acquire geometric invariance.”
LightGlue: Local Feature Matching at Light Speed (2023, methodology): “While classical algorithms rely on hand-crafted criteria and gradient statistics [41, 23, 4, 53], much of the recent research has focused on designing Convolutional Neural Networks (CNNs) for both detection [81, 16, 19, 52, 73] and description [42, 72].”
Unifying Deep Local and Global Features for Image Search (2020, methodology): “8 SOSNet [57] 45.”
Spatial-Aware Feature Aggregation for Image based Cross-View Geo-Localization (2019, background): “Due to the drastic appearance and viewpoint changes, it is very difficult to match local features [12, 2, 18, 22] between ground and aerial images directly.”
R2D2: Repeatable and Reliable Detector and Descriptor (2019, methodology): “As in previous works [1, 16, 25, 57, 58], we cast descriptor matching as a metric learning problem.”