FPNN: Field Probing Neural Networks for 3D Data

By Yangyan Li, Sören Pirk, Hao Su, Charles Ruizhongtai Qi, Leonidas J. Guibas. 2016

302 citations recorded by the source corpus.

Building discriminative representations for 3D data has been an important task in computer graphics and computer vision research. Convolutional Neural Networks (CNNs) have shown to operate on 2D images with great success for a variety of tasks. Lifting convolution operators to 3D (3DCNNs) seems like a plausible and promising next step. Unfortunately, the computational complexity of 3D CNNs grows cubically with respect to voxel resolution. Moreover, since most 3D geometry representations are boundary based, occupied regions do not increase proportionately with the size of the discretization, resulting in wasted computation. In this work, we represent 3D spaces as volumetric fields, and propose a novel design that employs field probing filters to efficiently extract features from them. Each field probing filter is a set of probing points --- sensors that perceive the space. Our learning algorithm optimizes not only the weights associated with the probing points, but also their locations, which deforms the shape of the probing filters and adaptively distributes them in 3D space. The optimized probing points sense the 3D space "intelligently", rather than operating blindly over the entire domain. We show that field probing is significantly more efficient than 3DCNNs, while providing state-of-the-art performance, on classification tasks for 3D object recognition benchmark datasets.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation (2016, methodology): “FPNN [12] and Vote3D [23] proposed special methods to deal with the sparsity problem; however, their operations are still on sparse volumes, it’s challenging for them to process very large point clouds.”
OctNet: Learning Deep 3D Representations at High Resolutions (2016, background): “[27] proposed field probing networks which sample 3D data at sparse points before feeding them into fully connected layers.”
Multi-View Silhouette and Depth Decomposition for High Resolution 3D Object Representation (2018, background): “Deep Learning with 3D Data Recent advances with 3D data have leveraged deep learning, beginning with architectures such as 3D convolutions [25, 19] for object classification.”
OctNetFusion: Learning Depth Fusion from Data (2017, background): “However, 3D data is often sparse in nature [25].”
Learning a Predictable and Generative Vector Representation for Objects (2016, methodology): “Past works have used part-based models [2, 16] and deep networks [39, 20, 24] for representing 3D models.”
PointCNN: Convolution On X-Transformed Points (2018, background): “Sparse kernels are proposed in [28], but this approach cannot be applied recursively for learning hierarchical features.”
SurfNet: Generating 3D Shape Surfaces Using Deep Residual Networks (2017, background): “The authors of ﬁeld probing neural networks [18] address the sparse oc-1 cupancy of voxel representations by developing adaptive 3D ﬁlters to reduce the cubic learning complexity.”
Distilling Knowledge From Graph Convolutional Networks (2020, methodology): “Deep learning based methods [39, 11, 7, 20, 29, 26] outperform the previous methods that based on hand-crafted feature extractors [1, 3].”
Frustum PointNets for 3D Object Detection from RGB-D Data (2017, methodology): “[16, 25, 32, 7] design more efficient 3D CNN or neural network architectures that exploit sparsity in point cloud.”
Point Transformer (2020, background): “To address these limitations, multiple extensions have been proposed that try to leverage the sparsity of 3D data [17,18,19], but still fail to process large amounts of input points.”
Mining Point Cloud Local Structures by Kernel Correlation and Graph Pooling (2017, unknown): “Only then do the 3D convolutional networks learn to extract features from voxels [2,7,23,25,30,42].”
SO-Net: Self-Organizing Network for Point Cloud Analysis (2018, methodology): “However, these methods still rely on uniform voxel grids and experience various limitations such as the lack of parallelization capacity [21].”
MeshNet: Mesh Neural Network for 3D Shape Representation (2018, methodology): “Field probing neural networks (FPNN) (Li et al. 2016), Vote3D (Wang and Posner 2015) and Octree-based convolutional neural network (OCNN) (Wang et al. 2017) address the sparsity problem, while they are still restricted with input getting larger.”
FoldingNet: Interpretable Unsupervised Learning on 3D Point Clouds (2017, background): “Applications of learning on point clouds include shape completion and recognition [54], unmanned autonomous vehicles [34], 3D object detection, recognition and classification [9, 31, 38, 39, 45, 46, 50], contour detection [20], layout inference [16], scene labeling [29], category discovery [57], poi”
Grasping Field: Learning Implicit Representations for Human Grasps (2020, background): “Learning 3D object models using various types of representations has also been explored [9, 23, 33, 44, 50, 52, 82, 90, 91].”
Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Network (2018, background): “Several improvements (Li et al. 2016; Wang and Posner 2015) have been proposed to relief the sparsity problem of the volumetric representation.”
FoldingNet: Point Cloud Auto-Encoder via Deep Grid Deformation (2017, background): “Applications of learning on point clouds include shape completion and recognition [57], unmanned autonomous vehicles [36], 3D object detection, recognition and classification [9, 33, 40, 41, 48, 49, 53], contour detection [21], layout inference [18], scene labeling [31], category discovery [60], poi”
Tangent Convolutions for Dense Prediction in 3D (2018, background): “[28] introduce Field Probing Neural Networks which respect the underlying sparsity of 3D data and are used for efficient feature extraction.”
Recurrent Slice Networks for 3D Segmentation of Point Clouds (2018, background): “[15] tried to reduce computation by sampling 3D data at sparse points before feeding them into networks.”
Escape from Cells: Deep Kd-Networks for the Recognition of 3D Point Cloud Models (2017, methodology): “To improve the scalability [34, 16] have considered sparse ways to define convolutions, while still using uniform 3D grids for representations.”