ASLFeat: Learning Local Features of Accurate Shape and Localization

By Zixin Luo, Lei Zhou, Xuyang Bai, Hongkai Chen, Jiahui Zhang, Yao Yao, Shiwei Li, Tian Fang, Long Quan. Computer Vision and Pattern Recognition, 2020

358 citations recorded by the source corpus.

This work focuses on mitigating two limitations in the joint learning of local feature detectors and descriptors. First, the ability to estimate the local shape (scale, orientation, etc.) of feature points is often neglected during dense feature extraction, while the shape-awareness is crucial to acquire stronger geometric invariance. Second, the localization accuracy of detected keypoints is not sufficient to reliably recover camera geometry, which has become the bottleneck in tasks such as 3D reconstruction. In this paper, we present ASLFeat, with three light-weight yet effective modifications to mitigate above issues. First, we resort to deformable convolutional networks to densely estimate and apply local transformation. Second, we take advantage of the inherent feature hierarchy to restore spatial resolution and low-level details for accurate keypoint localization. Finally, we use a peakiness measurement to relate feature responses and derive more indicative detection scores. The effect of each modification is thoroughly studied, and the evaluation is extensively conducted across a variety of practical scenarios. State-of-the-art results are reported that demonstrate the superiority of our methods.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/cf2541b047586e9ad499c171ed386f2a5b6dfdfe

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

1. Yohann Cabon
2. Vincent Leroy
3. Jérôme Revaud
4. Xingyu Jiang
5. Jiayi Ma
6. Zixin Luo
7. Junjun Jiang
8. Tian Fang

In Computer Vision and Pattern Recognition

1. Jiayi Ma
2. Yohann Cabon
3. Vincent Leroy
4. Jérôme Revaud
5. Xingyu Jiang
6. Luc Van Gool
7. Junchi Yan
8. Junjun Jiang

Citations in the corpus

• LoFTR: Detector-Free Local Feature Matching with Transformers (2021, background): “Many learning-based local features along this line [32, 11, 25, 28, 47] also adopt the detectorbased design.”
• Grounding Image Matching in 3D with MASt3R (2024, methodology): “Departing from the former handcrafted methods like SIFT [52,71], modern approaches have been shifting towards learning-based data-driven schemes for detecting keypoints [8,60,97,117], describing them [7,33, 37,88] or both at the same time [10,21,53,54,70,98].”
• ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformer (2022, methodology): “As is shown in previous works [10,8,9,12,28,27,26,29], CNN exhibits strong capability to capture local context and generates high-level features, which can be directly used to perform nearest neighbour matching.”
• Image Matching from Handcrafted to Deep Features: A Survey (2020, methodology): “Luo et al. (2020) proposed ASFeat to explore local shape information of feature points and enhance the accuracy of points detection, by jointly learning local feature detectors and descriptors.”
• BlendedMVS: A Large-Scale Dataset for Generalized Multi-View Stereo Networks (2019, methodology): “…it is also applicable to a variety of 3D geometry related tasks, including, but not limited to 1) image retrieval [28], 2) Image feature detection and description [17, 16, 18], 3) single-view/multi-view camera pose regression, 4) single-view depth/normal estimation and 5) disparity estimation.”