3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registration

By Zi Jian Yew, Gim Hee Lee. European Conference on Computer Vision, 2018

386 citations recorded by the source corpus.

In this paper, we propose the 3DFeat-Net which learns both 3D feature detector and descriptor for point cloud matching using weak supervision. Unlike many existing works, we do not require manual annotation of matching point clusters. Instead, we leverage on alignment and attention mechanisms to learn feature correspondences from GPS/INS tagged 3D point clouds without explicitly specifying them. We create training and benchmark outdoor Lidar datasets, and experiments show that 3DFeat-Net obtains state-of-the-art performance on these gravity-aligned datasets.

Subfields: Aerospace Engineering, Environmental Engineering, Geology

View Paper: https://www.semanticscholar.org/paper/8c1a17cd1827b00f40517b3c2e73e6b925e53faa

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Environmental Engineering

In Geology

Citations in the corpus

Deep Closest Point: Learning Representations for Point Cloud Registration (2019, methodology): “Recently, descriptor learning methods have brought significant progress in point cloud registration: 3DMatch [59] proposes learning a local volumetric patch descriptor to establish correspondences; 3DFeatNet [56] takes similar approach to point cloud representation for local regions; PPFFoldNet [9] ”
A comprehensive survey on point cloud registration (2021, methodology): “Average Recall Thresholds FPFH [86] 67 TE(2m),RE(5 ◦ ) USC [93] 100 TE(2m),RE(5 ◦ ) CGF [50] 92.1 TE(2m),RE(5 ◦ ) 3DMatch [114] 33.3 TE(2m),RE(5 ◦ ) 3DFeatNet [110] 95.2 TE(2m),RE(5 time complexity is huge, which remain at infancy.”
D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features (2020, methodology): “One attempt made by 3DFeat-Net [14] predicts a patch-wise detection score, whereas only limited spatial context is considered and a dense inference for the entire point cloud is not applicable in practice.”
RPM-Net: Robust Point Matching Using Learned Features (2020, background): “In addition to predicting feature descriptors, 3DFeat-Net [41] and USIP [18] also learn to detect salient keypoints.”
Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review (2020, background): “There also exist deep learning-based 3D descriptors (Zeng et al. 2017a; Yew and Lee 2018) aiming at matching 3D points, which are representative and discriminative.”
The Perfect Match: 3D Point Cloud Matching With Smoothed Densities (2018, methodology): “Since achieving the rotation invariance in practice is non-trivial, several recent works [49, 5, 45] choose to ignore it and thus do not generalize to rigidly transformed scenes [4].”
PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency (2021, methodology): “Although 3D local features [4, 39, 18, 27, 32] have evolved rapidly, correspondences produced by feature matching are still prone to outliers, especially when the overlap of scene fragments is small.”
Geometric Transformer for Fast and Robust Point Cloud Registration (2022, methodology): “5(top), we compare to the state-of-the-art RANSAC-based methods: 3DFeat-Net [34], FCGF [7], D3Feat [4], SpinNet [1], Predator [15] and CoFiNet [36].”
SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration (2020, methodology): “The registration is regarded as successful if the RTE and RRE of a pair of fragments are both below the predefined thresholds 2m and 5 in [60].”
Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Review (2020, background): “[145] or unsupervised learning [146, 147] should be developed to increase the model’s generalization and solve the data absence problem.”
Fully Convolutional Geometric Features (2019, background): “Yew and Lee [34] use a PointNet to extract features in outdoor scenes.”
PREDATOR: Registration of 3D Point Clouds with Low Overlap (2020, methodology): “Comparision to the SoTAs: We compare PREDATOR to 3DFeat-Net [47], FCGF [8] and D3Feat* [3]5 As shown in Tab.”
DeepVCP: An End-to-End Deep Neural Network for Point Cloud Registration (2019, methodology): “Inspired by the attention layer in 3DFeatNet [46], we design a point weighting layer to learn the saliency of each point in an end-to-end framework.”
CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration (2021, methodology): “On KITTI, we compare CoFiNet to 3DFeat-net [43], FCGF [31], D3Feat [10] and PREDA-TOR [12].”