Deep Parametric Continuous Convolutional Neural Networks

By Shenlong Wang, Simon Suo, Wei-Chiu Ma, Andrei Pokrovsky, Raquel Urtasun. 2018

494 citations recorded by the source corpus.

Standard convolutional neural networks assume a grid structured input is available and exploit discrete convolutions as their fundamental building blocks. This limits their applicability to many real-world applications. In this paper we propose Parametric Continuous Convolution, a new learnable operator that operates over non-grid structured data. The key idea is to exploit parameterized kernel functions that span the full continuous vector space. This generalization allows us to learn over arbitrary data structures as long as their support relationship is computable. Our experiments show significant improvement over the state-of-the-art in point cloud segmentation of indoor and outdoor scenes, and lidar motion estimation of driving scenes.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Environmental Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Environmental Engineering

Citations in the corpus

Dynamic Graph CNN for Learning on Point Clouds (2018, methodology): “2017], self-driving vehicles [Liang et al. 2018; Qi et al. 2017a; Wang et al. 2018b], robotics [Rusu et al.”
Swin Transformer V2: Scaling Up Capacity and Resolution (2021, methodology): “Our Log-CPB approach is also related to earlier works on continuous convolution and variants [30, 45, 58, 67], which utilize a meta network to handle irregular data points.”
PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Clouds (2021, background): “In addition, [41, 11, 58, 50, 52, 29] propose to directly learn the kernel of local points based on point positions.”
PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding (2020, background): “Finally, since the points are discrete samples of an underlying surface, continuous convolutions have also been considered [66, 4, 75].”
StarNet: Targeted Computation for Object Detection in Point Clouds (2019, background): “On-going efforts have attempted to design models that operate directly on point cloud data, some of which are derived to mimic convolutions [41, 49].”
4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks (2019, background): “86 SuperpointGraph [14] 58.”
3D Object Detection with Pointformer (2020, background): “PCCN [33] generalizes convolution to non-grid structured data by exploiting parameterized kernel functions that span the full continuous vector space.”
Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges (2019, methodology): “[141] propose a new learnable operator called Parametric Continuous Convolution to aggregate points via a weighted sum, and Li et al.”
Campus3D: A Photogrammetry Point Cloud Benchmark for Hierarchical Understanding of Outdoor Scene (2020, background): “Categories of pointbased deep learning models mainly include feature pooling models [15, 23, 24, 43], convolution-based models [20, 30, 34], graph-based models [18, 33, 37] and attention-based models [41, 42].”
LanczosNet: Multi-Scale Deep Graph Convolutional Networks (2019, background): “Several attention mechanisms have been introduced in [33, 34] to learn the weights over edges for GCNs. Notably, [8] proposes diffusion convolutional neural networks (DCNN) which uses diffusion operator for graph convolution.”
Multi-Task Multi-Sensor Fusion for 3D Object Detection (2019, methodology): “To overcome this drawback, ContFuse [13] uses continuous convolution [30] to fuse multi-scale convolutional feature maps from each sensor, where the correspondence between image and BEV spaces is achieved through projection of the LiDAR points.”
Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review (2020, background): “This point-wise fusion is achieved with continuous convolutions [83] fusion layers that bridge image and point cloud features of different scales at multiple stages in the network.”
FlowNet3D: Learning Scene Flow in 3D Point Clouds (2018, background): “[31] mentioned a larger dataset however it belongs to Uber and is not publicly available.”
PointASNL: Robust Point Clouds Processing Using Nonlocal Neural Networks With Adaptive Sampling (2020, methodology): “D vectors as edge relationship, PCCN [40] and PointConv [44] project the relative position of two points to a convolution weight.”
PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation (2020, background): “Other approaches [51, 48, 57, 52, 21] enhance feature learning on local regions by dynamic context aggregation and attention modules.”