PointNetLK: Robust & Efficient Point Cloud Registration Using PointNet

By Yasuhiro Aoki, Hunter Goforth, Rangaprasad Arun Srivatsan, Simon Lucey. Computer Vision and Pattern Recognition, 2019

920 citations recorded by the source corpus.

PointNet has revolutionized how we think about representing point clouds. For classification and segmentation tasks, the approach and its subsequent variants/extensions are considered state-of-the-art. To date, the successful application of PointNet to point cloud registration has remained elusive. In this paper we argue that PointNet itself can be thought of as a learnable "imaging" function. As a consequence, classical vision algorithms for image alignment can be brought to bear on the problem -- namely the Lucas & Kanade (LK) algorithm. Our central innovations stem from: (i) how to modify the LK algorithm to accommodate the PointNet imaging function, and (ii) unrolling PointNet and the LK algorithm into a single trainable recurrent deep neural network. We describe the architecture, and compare its performance against state-of-the-art in several common registration scenarios. The architecture offers some remarkable properties including: generalization across shape categories and computational efficiency -- opening up new paths of exploration for the application of deep learning to point cloud registration. Code and videos are available at https://github.com/hmgoforth/PointNetLK.

Subfields: Aerospace Engineering, Computational Mechanics, Geology

View Paper: https://www.semanticscholar.org/paper/325ffdedab735ac403f06832ca6bbb3539aa4bef

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computational Mechanics

In Geology

Citations in the corpus

TEASER: Fast and Certifiable Point Cloud Registration (2020, methodology): “PointNetLK [88] uses PointNet to learn feature representations and then iteratively align the features representations, instead of the 3-D coordinates.”
Deep Closest Point: Learning Representations for Point Cloud Registration (2019, methodology): “We train and test our model endto-end on ModelNet40 [52] in various settings, showing our model is not only efficient but also outperforms ICP and its extensions, as well as the recently-proposed PointNetLK method [18].”
Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review (2020, neutral): “, 2019b], DCP [Wang and Solomon, 2019a], PointNetLK [Aoki et al., 2019], PRNet [Wang and Solomon, 2019b], DeepICP [Lu et al.”
A comprehensive survey on point cloud registration (2021, methodology): “There is also some combination of conventional registration optimization strategies and deep learning techniques in an end-to-end framework [40], [16], [3], [99].”
Deep Global Registration (2020, methodology): “In other words, we can detect when our system might fail before it returns a result and fall back to a more accurate but time-consuming algorithm, unlike previous end-to-end methods that use globally pooled latent features [2] or a singly stochastic matrix [41] – such latent representations are more”
Feature-Metric Registration: A Fast Semi-Supervised Approach for Robust Point Cloud Registration Without Correspondences (2020, methodology): “To efﬁciently compute the Jacobian matrix, instead of calculating the Jacobian matrix by stochastic gradient approach, refer to [1], we utilise a different ﬁnite gradient to compute the Jacobian matrix: where ξ = ( R i , t i ) is the inﬁnitesimal perturbations of the transformation parameters…”
CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration (2021, methodology): “The second [17, 18] includes the correspondence-free methods based on the intuition that the feature distance between two well-aligned point clouds should be small.”
PRNet: Self-Supervised Learning for Partial-to-Partial Registration (2019, background): “Recently, PointNetLK [1] and Deep Closest Point (DCP) [2] show that learning-based registration can be faster and more robust than classical methods, even when trained on different datasets.”
PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency (2021, background): “Besides, end-toend registration networks [3, 62, 63, 71] have been proposed.”
PREDATOR: Registration of 3D Point Clouds with Low Overlap (2020, methodology): “PointNetLK [1] combines a PointNetbased global feature descriptor [28] with a Lucas/Kanadelike optimization algorithm [24] and estimates the relative transformation in an iterative fashion.”