PointNetVLAD: Deep Point Cloud Based Retrieval for Large-Scale Place Recognition

By Mikaela Angelina Uy, Gim Hee Lee. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

702 citations recorded by the source corpus.

Unlike its image based counterpart, point cloud based retrieval for place recognition has remained as an unexplored and unsolved problem. This is largely due to the difficulty in extracting local feature descriptors from a point cloud that can subsequently be encoded into a global descriptor for the retrieval task. In this paper, we propose the PointNetVLAD where we leverage on the recent success of deep networks to solve point cloud based retrieval for place recognition. Specifically, our PointNetVLAD is a combination/modification of the existing PointNet and NetVLAD, which allows end-to-end training and inference to extract the global descriptor from a given 3D point cloud. Furthermore, we propose the "lazy triplet and quadruplet" loss functions that can achieve more discriminative and generalizable global descriptors to tackle the retrieval task. We create benchmark datasets for point cloud based retrieval for place recognition, and the experimental results on these datasets show the feasibility of our PointNetVLAD. Our code and datasets are publicly available on the project website1.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Geology

View Paper: https://www.semanticscholar.org/paper/e69fa2ec8a83fd256f7e5843dc31c125d90360dd

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Geology

Citations in the corpus

Deep Learning vs. Traditional Computer Vision (2019, methodology): “e reference map. In pose estimation, the exact pose of the query image calculated more precisely with algorithms such as the Perspective-n-Point (PnP) [13] and geometric verification [18] algorithms. [56] The success of image based place recognition is largely attributed to the ability to extract im”
Scan Context++: Structural Place Recognition Robust to Rotation and Lateral Variations in Urban Environments (2021, methodology): “PointNetVLAD [26] leveraged a learning-based approach to summarize a place into a single vector representation.”
MulRan: Multimodal Range Dataset for Urban Place Recognition (2020, methodology): “In particular, LiDARbased place recognition methods [3, 14, 15, 2, 29, 30] have been widely studied to exploit the accuracy of direct 3D measurement for distant structures (e.”
LPD-Net: 3D Point Cloud Learning for Large-Scale Place Recognition and Environment Analysis (2018, methodology): “Similar to [22], the loss function of the network uses lazy quadruplet loss based on metric learning, so that the positive sample distance is reduced during the training process and the negative sample distance is enlarged to obtain a unique scene description vector.”
Zero-shot Learning of 3D Point Cloud Objects (2019, methodology): “Also, instead of Maxpooling, [24] used NetVlad [1] as a pooling mechanism to remove permutation from point cloud features.”
Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition (2021, background): “More recent approaches include ranking-loss based learning [59], novel pooling [58], contextual feature reweighting [40], large scale re-training [82], semantics-guided feature aggregation [28, 64, 75], use of 3D [53, 81, 43], additional sensors [32, 55, 22], sequences [25, 82], and image appearance”