FoldingNet: Point Cloud Auto-Encoder via Deep Grid Deformation

By Yaoqing Yang, Chen Feng, Yiru Shen, Dong Tian. 2017

1,401 citations recorded by the source corpus.

Recent deep networks that directly handle points in a point set, e.g., PointNet, have been state-of-the-art for supervised learning tasks on point clouds such as classification and segmentation. In this work, a novel end-to-end deep auto-encoder is proposed to address unsupervised learning challenges on point clouds. On the encoder side, a graph-based enhancement is enforced to promote local structures on top of PointNet. Then, a novel folding-based decoder deforms a canonical 2D grid onto the underlying 3D object surface of a point cloud, achieving low reconstruction errors even for objects with delicate structures. The proposed decoder only uses about 7% parameters of a decoder with fully-connected neural networks, yet leads to a more discriminative representation that achieves higher linear SVM classification accuracy than the benchmark. In addition, the proposed decoder structure is shown, in theory, to be a generic architecture that is able to reconstruct an arbitrary point cloud from a 2D grid. Our code is available at http://www.merl.com/research/license#FoldingNet

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Geology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Geology

Citations in the corpus

Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data (2019, neutral): “Alternatively, some other methods proposed to learn local features from convo-lutions, e.g ., [32, 25, 20, 42, 44, 18, 2, 24, 33, 11] or from autoencoders [45].”
Dynamic Graph CNN for Learning on Point Clouds (2018, methodology): “pe completion; a similar architecture was used by Ranjan et al. [2018] for 3D face synthesis. For point clouds, multiple generative architectures have been proposed [Fan et al. 2017; Li et al. 2018b; Yang et al. 2018]. 3 OUR APPROACH We propose an approach inspired by PointNet and convolution operat”
Learning Implicit Fields for Generative Shape Modeling (2018, background): “There have been a variety of 3D shape representations for deep learning of shapes, such as voxel grids [9, 14, 28, 45, 46], octrees [18, 35, 40, 43, 44], multi-view images [29, 39], point clouds [1, 12, 13, 32, 33, 47, 48], geometry images [37, 38], deformable mesh/patches [16, 38, 42, 47], and part”
Diffusion Probabilistic Models for 3D Point Cloud Generation (2021, background): “FoldingNet and At-lasNet [26, 10] mitigate this issue by learning a mapping from 2D patches to the 3D space, which deforms the 2D patches into the shape of point clouds.”
Learning Gradient Fields for Shape Generation (2020, neutral): “N×3 where N is predefined) [2,18,67,56,68,17,52,59], or relies on heuristic set distance functions such as Chamfer distance and Earth Mover Distance [25,65,19,12,6].”
Distilling Knowledge From Graph Convolutional Networks (2020, methodology): “Moreover, GCN based methods [8, 19, 37, 18, 41], which can directly encode the structure information from the set of points, become one of the most popular directions along this line.”
Rotation Invariant Convolutions for 3D Point Clouds Deep Learning (2019, methodology): “Some methods exploit hierarchical structures and clustering for scalability [28, 15, 33, 34, 16], mapping point cloud to two dimensional space [40, 9, 18], applying spectral analysis [42], or addressing non-uniform point distribution [10].”
Patch-Based Progressive 3D Point Set Upsampling (2018, methodology): “Our feature expansion method is also related to recent point cloud generative models FoldingNet [56] and AtlasNet [13], where the coordinates of a 2D point are attached to the learned features for point generation.”
Mining Point Cloud Local Structures by Kernel Correlation and Graph Pooling (2017, positive): “Yet it is non-trivial to generalize from mesh to point clouds [43].”
3D Shape Generation and Completion through Point-Voxel Diffusion (2021, background): “Many prior works have explored point cloud generation in terms of autoencoding [1, 11, 47], single-view reconstruction [12, 10, 18, 17], and adversarial generation [34, 48, 40].”
Shape Prior Deformation for Categorical 6D Object Pose and Size Estimation (2020, result): “Similar clustering results are also observed on a different set of models [39].”
PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding (2020, methodology): “3D tasks are potentially the biggest beneciaries of unsupervisedand transfer-learning. This was shown in several works on single object tasks like reconstruction, classication and part segmentation [1, 16, 74, 21, 36, 67, 22, 53]. Yet, generally much less attention has been devoted to representation”
Shape As Points: A Differentiable Poisson Solver (2021, background): “Explicit shape representations and learning algorithms depending on such representations directly parameterize the surface of the geometry, either as a point cloud [17, 40, 55, 56, 67, 70], parameterized mesh [22, 26, 29, 64] or surface patches [2,20,39,45,68,71,72].”
Point Cloud GAN (2018, background): “The difference between it and the point cloud autoencoder (Achlioptas et al., 2017; Yang et al., 2018) will be discussed in Section 5.”
Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Clouds (2020, methodology): “The baseline model A can be viewed as a variant of FoldingNet [55], which was trained by self-reconstruction loss only and gets a low classification accuracy of 86.”
Unsupervised Point Cloud Pre-training via Occlusion Completion (2020, methodology): “Work in generative modelling includes models based on generative adversarial networks (GANs) [54, 1, 14], autoencoders [15, 29, 59], normalizing ﬂows [58], and approximate convex decomposition [12].”