3D shape segmentation via shape fully convolutional networks

By Pengyu Wang, Y. Gan, Panpan Shui, Fenggen Yu, Yan Zhang, Song-Le Chen, Zhengxing Sun. 2017

92 citations recorded by the source corpus.

Abstract We propose a novel fully convolutional network architecture for shapes, denoted by Shape Fully Convolutional Networks (SFCN). 3D shapes are represented as graph structures in the SFCN architecture, based on novel graph convolution and pooling operations, which are similar to convolution and pooling operations used on images. Meanwhile, to build our SFCN architecture in the original image segmentation fully convolutional network (FCN) architecture, we also design and implement a generating operation with bridging function. This ensures that the convolution and pooling operation we have designed can be successfully applied in the original FCN architecture. In this paper, we also present a new shape segmentation approach based on SFCN. Furthermore, we allow more general and challenging input, such as mixed datasets of different categories of shapes which can prove the ability of our generalisation. In our approach, SFCNs are trained triangles-to-triangles by using three low-level geometric features as input. Finally, the feature voting-based multi-label graph cuts is adopted to optimise the segmentation results obtained by SFCN prediction. The experiment results show that our method can effectively learn and predict mixed shape datasets of either similar or different characteristics, and achieve excellent segmentation results.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

Citations in the corpus

Occupancy Networks: Learning 3D Reconstruction in Function Space (2018, methodology): “Mesh Representations: Meshes have ﬁrst been considered for discriminative 3D classiﬁcation or segmentation tasks by applying convolutions on the graph spanned by the mesh’s vertices and edges [5,27,71].”
Deep Learning for 3D Point Clouds: A Survey (2019, methodology): “[251] first performed shape segmentation on 3D meshes by introducing Shape Fully Convolutional Net-”
PointCLIP: Point Cloud Understanding by CLIP (2021, background): “Also, different from other off-line projection methods, whose projected images are generated from meshes [54] or CAD models [48], our projected depth maps are from raw points and contain no color information but scattered depth values, which leads to marginal time and computation cost.”
Deep Marching Cubes: Learning Explicit Surface Representations (2018, background): “Several works consider the problem of shape classification and segmentation by defining neural networks which operate on the graph spanned by the edges and vertices of a 3D mesh [3, 14, 43].”
FoldingNet: Point Cloud Auto-Encoder via Deep Grid Deformation (2017, background): “Applications of learning on point clouds include shape completion and recognition [57], unmanned autonomous vehicles [36], 3D object detection, recognition and classification [9, 33, 40, 41, 48, 49, 53], contour detection [21], layout inference [18], scene labeling [31], category discovery [60], poi”
FoldingNet: Interpretable Unsupervised Learning on 3D Point Clouds (2017, background): “Applications of learning on point clouds include shape completion and recognition [54], unmanned autonomous vehicles [34], 3D object detection, recognition and classification [9, 31, 38, 39, 45, 46, 50], contour detection [20], layout inference [16], scene labeling [29], category discovery [57], poi”