KPConv: Flexible and Deformable Convolution for Point Clouds
By Hugues Thomas, Charles Ruizhongtai Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, Leonidas J. Guibas. 2019
3,358 citations recorded by the source corpus.
We present Kernel Point Convolution (KPConv), a new design of point convolution, i.e. that operates on point clouds without any intermediate representation. The convolution weights of KPConv are located in Euclidean space by kernel points, and applied to the input points close to them. Its capacity to use any number of kernel points gives KPConv more flexibility than fixed grid convolutions. Furthermore, these locations are continuous in space and can be learned by the network. Therefore, KPConv can be extended to deformable convolutions that learn to adapt kernel points to local geometry. Thanks to a regular subsampling strategy, KPConv is also efficient and robust to varying densities. Whether they use deformable KPConv for complex tasks, or rigid KPconv for simpler tasks, our networks outperform state-of-the-art classification and segmentation approaches on several datasets. We also offer ablation studies and visualizations to provide understanding of what has been learned by KPConv and to validate the descriptive power of deformable KPConv.
Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Geology, 3D Vision & Pose
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computational Mechanics
- Or Litany
- Matthias Nießner
- Jiaya Jia
- Li Jiang
- Leonidas J. Guibas
- Shaoshuai Shi
- Vladlen Koltun
- Gerard Pons-Moll
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Xiaogang Wang
- Dacheng Tao
- Ming-Hsuan Yang
- Trevor Darrell
- Jitendra Malik
- Andrew Zisserman
- Yi Yang
In Geology
In 3D Vision & Pose
- Matthias Nießner
- Or Litany
- Angela Dai
- Andreas Geiger
- Marc Pollefeys
- Gerard Pons-Moll
- Jiajun Wu
- Silvio Savarese
Citations in the corpus
- Dynamic Neural Networks: A Survey (2021, background): “Point Cloud Sa & Sp 3D Shape classification and segmentation, 3D scene segmentation [240], 3D semantic scene completion [241]”
- Intrinsic-Extrinsic Convolution and Pooling for Learning on 3D Protein Structures (2021, methodology): “While we state our operation as an unstructured convolution as done in 3D point cloud learning (Hermosilla et al., 2018; Groh et al., 2018; Wu et al., 2019; Xiong et al., 2019; Thomas et al., 2019), it can also be understood in a message passing framework (Kipf & Welling, 2017) where hidden states a”
- Revealing Occlusions with 4D Neural Fields (2022, background): “Because existing 4D architectures [13, 29, 56] lack a mechanism to efficiently create new points, they have not been demonstrated to be capable of dynamic scene completion.”
- 2DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Clouds (2022, background): “Later on, many studies design point-wise MLP [25,26], adaptive weight [27, 28] and pseudo grid [29, 30] based methods to extract local features of point clouds or exploit nonlocal operators [31–33] to learn long distance dependency.”
- PRNet: Self-Supervised Learning for Partial-to-Partial Registration (2019, methodology): “Other works, including SPLATNet [30], SplineCNN [31], KPConv [32], and GWCNN [33], transform 3D shapes to regular grids for feature learning.”
- Learning 3D Semantic Scene Graphs From 3D Indoor Reconstructions (2020, background): “Current 3D scene understanding works include perception tasks such as instance segmentation [12, 21, 44, 50], semantic segmentation [34, 36, 5, 38] as well as 3D object detection and classification [40, 34, 35, 54].”
- PREDATOR: Registration of 3D Point Clouds with Low Overlap (2020, methodology): “We follow [35] and first down-sample raw point clouds with a voxel-grid filter of size V , such that P and Q have reasonably uniform point density.”
- Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds (2019, methodology): “Another pipeline is convolutional kernel based approaches [55; 27; 47].”
- Under review as a conference paper at ICLR 2020 completely free-form sampling in image space : Spatial Transformers (2019, background): “While kernel space sampling is also studied in Deformable Filter (Xiong et al., 2019) and KPConv (Thomas et al., 2019), but in their contexts, sampling grids are computed from input point clouds rather than learned from data corpora.”
- Under review as a conference paper at ICLR 2020 completely free-form sampling in image space : Spatial Transformers (2019, background): “While kernel space sampling is also studied in Deformable Filter (Xiong et al., 2019) and KPConv (Thomas et al., 2019), but in their contexts, sampling grids are computed from input point clouds rather than learned from data corpora.”
- PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection (2019, background): “Representation learning on point clouds has drawn lots of attention on improving the performance of point cloud classification and segmentation [25, 26, 45, 34, 7, 42, 16, 30, 36, 8, 32, 10, 21, 3].”
- Mask3D: Mask Transformer for 3D Semantic Instance Segmentation (2022, background): “A major step forward was sparked by powerful feature backbones [17, 53, 57] such as sparse convolutional networks [8, 16] that improve over existing PointNets [42, 44] and dense 3D CNNs [36, 43, 60].”
- SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration (2020, methodology): “First, many of these methods such as D3Feat [2] and FCGF [8] rely on kernel-based point convolution [53] or submanifold sparse convolution [23] to extract per-point features, resulting in the learned point local patterns being rotationally variant.”
- 3D Object Detection with Pointformer (2020, methodology): “Compared to the existing local feature extraction modules in [36, 37, 29], the proposed Local Transformer has several advantages.”
- Geometric Back-Projection Network for Point Cloud Classification (2019, neutral): “9 KP-Conv [66] coords 1k 92.”
- Compositional Prototype Network with Multi-view Comparision for Few-Shot Point Cloud Semantic Segmentation (2020, methodology): “There are mainly three categories in analysis methods for 3D point cloud: point-based methods [16–18], multi-view methods [19], and volumetric methods [20].”
- PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection (2021, background): “Representation Learning on Point Clouds Recently representation learning on point clouds has drawn lots of attention for improving the performance of 3D classification and segmentation (Qi et al., 2017a, b; Wang et al., 2019b; Huang et al., 2018; Zhao et al., 2019; Li et al., 2018; Su et al., 2018; ”
- RELLIS-3D Dataset: Data, Benchmarks and Analysis (2020, methodology): “Compared with SalsaNext, the degradation of KPConv is more obvious.”