LoFTR: Detector-Free Local Feature Matching with Transformers

By Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, Xiaowei Zhou. Computer Vision and Pattern Recognition, 2021

1,958 citations recorded by the source corpus.

We present a novel method for local image feature matching. Instead of performing image feature detection, description, and matching sequentially, we propose to first establish pixel-wise dense matches at a coarse level and later refine the good matches at a fine level. In contrast to dense methods that use a cost volume to search correspondences, we use self and cross attention layers in Transformer to obtain feature descriptors that are conditioned on both images. The global receptive field provided by Transformer enables our method to produce dense matches in low-texture areas, where feature detectors usually struggle to produce repeatable interest points. The experiments on indoor and outdoor datasets show that LoFTR outperforms state-of-the-art methods by a large margin. LoFTR also ranks first on two public benchmarks of visual localization among the published methods. Code is available at our project page: https://zju3dv.github.io/loftr/.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/b91de7d12ec1103f6ef9eb0720d697a9e7ecc9fe

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

VGGT: Visual Geometry Grounded Transformer (2025, neutral): “Two-view image matching is a widely-explored topic [47, 61, 69] in computer vision.”
Geometric Transformer for Fast and Robust Point Cloud Registration (2022, methodology): “Inspired by the recent advances in image matching [22, 25, 39], keypoint-free methods [36] downsample the input point clouds into superpoints and then match them through examining whether their local neighborhood (patch) overlaps.”
FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects (2023, methodology): “Table 1 presents the comparison results against the state-of-art RGBD methods [21, 27, 52] on YCB-Video dataset.”
Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation (2022, methodology): “Following [41], we add positional encoding to the backbone output, which enhances positional dependence of the feature maps.”
CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration (2021, methodology): “In a similar coarse-to-ﬁne manner with Patch2Pixel, LoFTR [25] leverages Transformers [33], together with an optimal transport matching layer [26], to match mutual-nearest patches on the coarse level, and then reﬁnes the corresponding pixel of the patch center on the ﬁner level.”
RoMa: Robust Dense Feature Matching (2023, positive): “Following previous work [12, 17, 44], use a model trained on the ScanNet training set when evaluating on ScanNet-1500.”
Continuous 3D Perception Model with Persistent State (2025, neutral): “…replacing hand-crafted components with learning-based alternatives (e.g., substituting conventional feature descriptors [6, 47] with learned ones [17, 20, 63, 73]), integrating data-driven priors [13, 77–79, 93, 97, 105] into the systems, or optimizing entire systems end-to-end [76, 78, 83, 94].”
One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization (2023, methodology): “Given a set of four predicted nearby views, we perform feature matching to identify corresponding keypoints across each pair of images (a total of six pairs) using an off-the-shelf module LoFTR [69].”
ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformer (2022, methodology): “As validated in Transformer networks [13,30,14], positional encoding is critical in maintaining spatial information for the flattened tokens.”
Grounding Image Matching in 3D with MASt3R (2024, methodology): “…has proven effective in scenarios where detailed spatial relationships and textures are critical for understanding scene geometry, leading to top performance on many benchmarks [4, 5, 6, 59, 72, 82] that are especially challenging for keypoints due to extreme changes in viewpoint or illumination.”
FlowFormer: A Transformer Architecture for Optical Flow (2022, methodology): “Visual correspondence tasks [45,19,7,27,53] is a main stream in computer vision.”
Driving Into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving (2023, methodology): “This metric utilizes a pre-trained matching model [51] to calculate the average number of matching key points, thereby quantifying the KPM score.”
Deep ViT Features as Dense Visual Descriptors (2021, methodology): “Recent works employ transformers for dense correspondence in intra-class pairs [8, 47, 22].”
TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers (2022, methodology): “Multi-head attention is a popular mechanism to perform information exchange and build a soft association between two sets of inputs, and it has been widely used for the feature matching task [34,41].”
GMFlow: Learning Optical Flow via Global Matching (2021, methodology): “The top-performing sparse methods (e.g., SuperGlue [34] and LoFTR [38]) adopt Transformers [41] to reason about the mutual relationship between feature descriptors, and the correspondences are extracted with an explicit matching layer (e.g., softmax layer [43]).”
DUSt3R: Geometric 3D Vision Made Easy (2023, positive): “These improvements encompass advanced feature description [26, 28, 97, 135, 167], more accurate image matching [3, 17, 60, 82, 100, 120, 126, 145], featuremetric refinement [59], and neural bundle adjustment [58, 153].”
LightGlue: Local Feature Matching at Light Speed (2023, methodology): “Following best practices in benchmarking [31] and unlike past works [56, 68], we use a state-of-the-art robust estimator [3] and extensively tune the inlier threshold for each method separately.”
BLINK: Multimodal Large Language Models Can See but Not Perceive (2024, methodology): “Surprisingly, we find that the Caption + LLM setting achieves better [70] DepthAnything [83] LoFTR [68] DIFT [70] DIRE [79] Ordinal Shading [14] 96.51 Effect of visual prompting on Blink : Several Blink tasks involve visual prompting.”