D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features

By Xuyang Bai, Zixin Luo, Lei Zhou, Hongbo Fu, Long Quan, Chiew-Lan Tai. Computer Vision and Pattern Recognition, 2020

531 citations recorded by the source corpus.

A successful point cloud registration often lies on robust establishment of sparse matches through discriminative 3D local features. Despite the fast evolution of learning-based 3D feature descriptors, little attention has been drawn to the learning of 3D feature detectors, even less for a joint learning of the two tasks. In this paper, we leverage a 3D fully convolutional network for 3D point clouds, and propose a novel and practical learning mechanism that densely predicts both a detection score and a description feature for each 3D point. In particular, we propose a keypoint selection strategy that overcomes the inherent density variations of 3D point clouds, and further propose a self-supervised detector loss guided by the on-the-fly feature matching results during training. Finally, our method achieves state-of-the-art results in both indoor and outdoor scenarios, evaluated on 3DMatch and KITTI datasets, and shows its strong generalization ability on the ETH dataset. Towards practical use, we show that by adopting a reliable feature detector, sampling a smaller number of features is sufficient to achieve accurate and fast point cloud alignment.

Subfields: Aerospace Engineering, Computational Mechanics, Geology

View Paper: https://www.semanticscholar.org/paper/231afc2fe804739c118a8d6dfaf16d27527493d8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

1. Lei Li
2. Zheng Qin
3. Kenji Koide
4. Lei Zhou
5. Zeyu Hu
6. Atsuhiko Banno
7. Shuji Oishi
8. Hongkai Chen

In Computational Mechanics

1. Lei Li
2. Zheng Qin
3. Kenji Koide
4. Lei Zhou
5. Zeyu Hu
6. Atsuhiko Banno
7. Shuji Oishi
8. Hongkai Chen

In Geology

1. Lei Li
2. Zheng Qin
3. Kenji Koide
4. Lei Zhou
5. Zeyu Hu
6. Atsuhiko Banno
7. Shuji Oishi
8. Hongkai Chen

Citations in the corpus

• SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration (2020, methodology): “To further evaluate the robustness of all approaches against rotations, we follow [14, 2] to build a rotated 3DMatch benchmark by applying arbitrary rotations in SO(3) group to all fragments of the dataset.”
• Voxelized GICP for Fast and Accurate 3D Point Cloud Registration (2020, background): “Several works proposed networks to extract local robust descriptors of input point clouds [13], [14], while some works proposed networks that directly perform regression of the relative pose between point clouds in the end-to-end fashion [15].”
• Geometric Transformer for Fast and Robust Point Cloud Registration (2022, methodology): “Following [4, 15], we evaluate the performance with three metrics: (1) Inlier Ratio (IR), the fraction of putative correspondences whose residuals are below a certain 3DMatch 3DLoMatch # Samples 5000 2500 1000 500 250 5000 2500 1000 500 250”
• PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency (2021, methodology): “Although 3D local features [4, 39, 18, 27, 32] have evolved rapidly, correspondences produced by feature matching are still prone to outliers, especially when the overlap of scene fragments is small.”