The Perfect Match: 3D Point Cloud Matching With Smoothed Densities

By Zan Gojcic, Caifa Zhou, Jan Dirk Wegner, Andreas Wieser. Computer Vision and Pattern Recognition, 2018

543 citations recorded by the source corpus.

We propose 3DSmoothNet, a full workflow to match 3D point clouds with a siamese deep learning architecture and fully convolutional layers using a voxelized smoothed density value (SDV) representation. The latter is computed per interest point and aligned to the local reference frame (LRF) to achieve rotation invariance. Our compact, learned, rotation invariant 3D point cloud descriptor achieves 94.9% average recall on the 3DMatch benchmark data set, outperforming the state-of-the-art by more than 20 percent points with only 32 output dimensions. This very low output dimension allows for near realtime correspondence search with 0.1 ms per feature point on a standard PC. Our approach is sensor- and scene-agnostic because of SDV, LRF and learning highly descriptive features with fully convolutional layers. We show that 3DSmoothNet trained only on RGB-D indoor scenes of buildings achieves 79.0% average recall on laser scans of outdoor vegetation, more than double the performance of our closest, learning-based competitors. Code, data and pre-trained models are available online at https://github.com/zgojcic/3DSmoothNet.

Subfields: Aerospace Engineering, Environmental Engineering, Geology

View Paper: https://www.semanticscholar.org/paper/7d2da48f8004f94ec80cea005c3713f514394f60

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Environmental Engineering

In Geology

Citations in the corpus

CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration (2021, methodology): “Differently, the last category of methods [6, 7, 8, 9, 10, 11, 12] tackles point cloud registration in a two-stage manner.”
Deep Global Registration (2020, methodology): “A number of recent methods are based on global pooling models [12, 11, 49], while others use convolutional networks [14, 9].”
A comprehensive survey on point cloud registration (2021, methodology): “Many existing experiments [35], [6], [41] show that directly apply the mathematical theories of registration will cost huge computation time, while directly apply deep learning will not guarantee accuracy.”
Deep Closest Point: Learning Representations for Point Cloud Registration (2019, methodology): “Recently, descriptor learning methods have brought significant progress in point cloud registration: 3DMatch [59] proposes learning a local volumetric patch descriptor to establish correspondences; 3DFeatNet [56] takes similar approach to point cloud representation for local regions; PPFFoldNet [9] ”
Geometric Transformer for Fast and Robust Point Cloud Registration (2022, background): “We ﬁrst compare the correspondence results of our method with the recent state of the arts: PerfectMatch [15], FCGF [8], D3Feat [4], SpinNet [1], Predator [16], YOHO [33] and CoFiNet [41] in Tab.”
DeepVCP: An End-to-End Deep Neural Network for Point Cloud Registration (2019, background): “We are also considering to try rotation invariant 3D descriptors [6, 16, 23] in the future.”
PREDATOR: Registration of 3D Point Clouds with Low Overlap (2020, methodology): “To alleviate artefacts caused by noise and voxelisation, [16] proposed to use a smoothed density voxel grid as input to a 3D CNN.”
SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration (2020, methodology): “We first compare the FMR scores achieved by our SpinNet and strong baselines (including LMVD [37], D3Feat [2], FCGF [8], PerfectMatch [22], PPFNet [14], and PPF-FoldNet [13]) on the 3DMatch dataset, under the conditions of sampling points f=5000, distance threshold τ1=10 cm and inlier ratio threshol”
Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review (2020, unknown): “[Gojcic et al., 2019], Yuan et al.”
Feature-Metric Registration: A Fast Semi-Supervised Approach for Robust Point Cloud Registration Without Correspondences (2020, methodology): “Figure 10 shows that the proposed method can handle partial overlap problem, and obtain bet-Ours Method LM-ICP [9] [1] Ours Rot.”
Voxelized GICP for Fast and Accurate 3D Point Cloud Registration (2020, background): “Several works proposed networks to extract local robust descriptors of input point clouds [13], [14], while some works proposed networks that directly perform regression of the relative pose between point clouds in the end-to-end fashion [15].”
D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features (2020, methodology): “For D3Feat, we report the results on both 6.25cm and 5cm to compare with PerfectMatch and FCGF, respectively.”
PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency (2021, methodology): “Although 3D local features [4, 39, 18, 27, 32] have evolved rapidly, correspondences produced by feature matching are still prone to outliers, especially when the overlap of scene fragments is small.”
Fully Convolutional Geometric Features (2019, methodology): “FCGF outperforms all hand-crafted features and PointNet-based methods by a large margin and marginally outperforms a recent 3D-convolution-based method [11].”
TEASER: Fast and Certifiable Point Cloud Registration (2020, methodology): “We use 3DSmoothNet [2], a state-of-the-art neural network, to compute local descriptors for each 3-D keypoint, and generate correspondences using nearest-neighbor search.”