Deep Closest Point: Learning Representations for Point Cloud Registration

By Yue Wang, Justin M. Solomon. IEEE International Conference on Computer Vision, 2019

1,065 citations recorded by the source corpus.

Point cloud registration is a key problem for computer vision applied to robotics, medical imaging, and other applications. This problem involves finding a rigid transformation from one point cloud into another so that they align. Iterative Closest Point (ICP) and its variants provide simple and easily-implemented iterative methods for this task, but these algorithms can converge to spurious local optima. To address local optima and other difficulties in the ICP pipeline, we propose a learning-based method, titled Deep Closest Point (DCP), inspired by recent techniques in computer vision and natural language processing. Our model consists of three parts: a point cloud embedding network, an attention-based module combined with a pointer generation layer to approximate combinatorial matching, and a differentiable singular value decomposition (SVD) layer to extract the final rigid transformation. We train our model end-to-end on the ModelNet40 dataset and show in several settings that it performs better than ICP, its variants (e.g., Go-ICP, FGR), and the recently-proposed learning-based method PointNetLK. Beyond providing a state-of-the-art registration technique, we evaluate the suitability of our learned features transferred to unseen objects. We also provide preliminary analysis of our learned model to help understand whether domain-specific and/or global features facilitate rigid registration.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/d4c1cfc0cce2140fb8cf5c175c0a34b467298ee9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Self-Supervised Pretraining of 3D Features on any Point-Cloud (2021, methodology): “Most methods on self-supervised learning focus on single 3D object representation with different applications to reconstruction, classification or part segmentation [1, 2, 25, 33, 35, 44, 49, 74, 99, 110].”
Point Transformer V3: Simpler, Faster, Stronger (2023, background): “While major efforts in 3D representation learning focused on individual objects [57, 68, 69, 87, 103], some recent advancements have redirected attention towards training on real-world scene-centric point clouds [30, 36, 91, 94, 107].”
SuperGlue: Learning Feature Matching With Graph Neural Networks (2019, background): “Works that learn to perform matching have so far focused on dense matching [43] or 3D point clouds [59], and still exhibit the same limitations.”
Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review (2020, neutral): “, 2019b], DCP [Wang and Solomon, 2019a], PointNetLK [Aoki et al.”
Deep Global Registration (2020, methodology): “Our method successfully aligns a challenging 3D pair ( left ), while RANSAC [35], FGR [50], and DCP [41] fail.”
RPM-Net: Robust Point Matching Using Learned Features (2020, methodology): “For PointNetLK and Deep Closest Point, we use the implementation provided by the authors but re-trained the networks since both works do not provide the required pretrained models 2 .”
PCT: Point cloud transformer (2020, methodology): “yer perceptron (MLP) splits and attentional splits. Unlike the above methods, our PCT is based on Transformer rather than using self-attention as an auxiliary module. While a framework by Wang et al. [28] uses Transformer to optimize point cloud registration, our PCT is a more general framework whic”
A comprehensive survey on point cloud registration (2021, background): “2 m ) DGR [16] 91.3 TE(0.3m),RE(15 ◦ ) PointNetLK [3] 1.61 TE(0.3m),RE(15 ◦ ) DCP [99] 3.22 TE(0.3m),RE(15”
CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration (2021, background): “The first [13, 14, 15] follows the idea of ICP [16], where they iteratively find dense correspondences and compute pose estimation.”
Metaverse: Perspectives from graphics, interactions and visualization (2022, background): “Recently, the developments of optimization-based methods and deep learning methods have improved registration robustness and efficiency (Huang et al., 2021; Perez-Gonzalez et al., 2019; Wang and Solomon, 2019).”
Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts (2020, methodology): “Following suit, recent works attempt to adapt the 2D pretext tasks to 3D, but mostly focus on single object classiﬁcation tasks on ShapeNet [1, 17, 70, 22, 37, 62, 26, 51, 50].”
PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding (2020, methodology): “In the past few years, the field of 3D deep learning has witnessed much progress with an ever-increasing number of 3D representation learning schemes [1, 16, 74, 21, 36, 67, 22, 15, 81, 12, 9].”
PRNet: Self-Supervised Learning for Partial-to-Partial Registration (2019, methodology): “For ease of comparison to previous work, we use the same notation as [2].”
PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency (2021, background): “Besides, end-toend registration networks [3, 62, 63, 71] have been proposed.”
Fast and Robust Iterative Closest Point (2020, methodology): “First, to obtain good initial alignment for challenging cases, we can potentially adopt a machine learning-based method for determining a coarse alignment; this is similar to the practice in [35] that uses ICP to reﬁne a DCP alignment.”
Geometric Transformer for Fast and Robust Point Cloud Registration (2022, methodology): “Cross-attention is a typical module for point cloud registration task [15, 31, 36], used to perform feature exchange between two input point clouds.”
PREDATOR: Registration of 3D Point Clouds with Low Overlap (2020, methodology): “9 shows that PREDATOR substantially increases the relative overlap and reduces the number of pairs with overlap 70% by more than 40 pp. Comparison to direct registration methods: To be able to compare PREDATOR to RPM-Net [48] and DCP [40], we resort to the synthetic, object-centric dataset they wer”