3D Mesh Labeling via Deep Convolutional Neural Networks

By Kan Guo, Dongqing Zou, Xiaowu Chen. 2015

230 citations recorded by the source corpus.

This article presents a novel approach for 3D mesh labeling by using deep Convolutional Neural Networks (CNNs). Many previous methods on 3D mesh labeling achieve impressive performances by using predefined geometric features. However, the generalization abilities of such low-level features, which are heuristically designed to process specific meshes, are often insufficient to handle all types of meshes. To address this problem, we propose to learn a robust mesh representation that can adapt to various 3D meshes by using CNNs. In our approach, CNNs are first trained in a supervised manner by using a large pool of classical geometric features. In the training process, these low-level features are nonlinearly combined and hierarchically compressed to generate a compact and effective representation for each triangle on the mesh. Based on the trained CNNs and the mesh representations, a label vector is initialized for each triangle to indicate its probabilities of belonging to various object parts. Eventually, a graph-based mesh-labeling algorithm is adopted to optimize the labels of triangles by considering the label consistencies. Experimental results on several public benchmarks show that the proposed approach is robust for various 3D meshes, and outperforms state-of-the-art approaches as well as classic learning algorithms in recognizing mesh labels.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Geology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

In Geology

Citations in the corpus

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation (2016, methodology): “Feature-based DNNs: [6, 8] firstly convert the 3D data into a vector, by extracting traditional shape features and then use a fully connected net to classify the shape.”
Occupancy Networks: Learning 3D Reconstruction in Function Space (2018, background): “Mesh Representations: Meshes have first been considered for discriminative 3D classification or segmentation tasks by applying convolutions on the graph spanned by the mesh’s vertices and edges [5, 27, 71].”
SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation (2016, methodology): “Along this track, most previous methods [12, 28, 18, 7] employ traditional machine learning techniques and construct classifiers based on geometric features.”
Distilling Knowledge From Graph Convolutional Networks (2020, methodology): “Deep learning based methods [39, 11, 7, 20, 29, 26] outperform the previous methods that based on hand-crafted feature extractors [1, 3].”
Deep Learning on Point Sets for 3 D Classification and Segmentation (2016, methodology): “Feature-based DNNs: [6, 8] firstly convert the 3D data into a vector, by extracting traditional shape features and then use a fully connected net to classify the shape.”
SEGCloud: Semantic Segmentation of 3D Point Clouds (2017, background): “cene Segmentation [13,2,40] but we focus mainly on the ones related to the above topics. Neural Networks for 3D Data: 3D Neural Networks have been extensively used for 3D object and parts recognition [60,54,46,25,53,21], understanding object shape priors, as well as generating and reconstructing obj”
Convolutional neural networks on surfaces via seamless toric covers (2017, methodology): “We compared our method to the state-of-the-art CNN method for mesh segmentation [Guo et al. 2015].”
GWCNN: A Metric Alignment Layer for Deep Shape Analysis (2017, unknown): “…representation for geometry is a 3D voxel grid, where each voxel stores a binary occupancy function [MS15] One can also directly analyze features of unordered surface elements such as mesh faces [GZC15], but this approach requires powerful features that provide additional contextual information.”
Attentional ShapeContextNet for Point Cloud Recognition (2018, background): “In areas outside of computer vision, the 3D shape classification and recognition problem has been extensively studied in computer graphics [7, 12, 28] and robotics [27, 36].”
A survey of deep neural network architectures and their applications (2017, background): “Towards this target, a more effective representation of meshes was introduced in [50].”
3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions (2016, methodology): “[14], which uses a 2D ConvNet descriptor to match local geometric features for mesh labeling.”
PointGrid: A Deep Network for 3D Shape Understanding (2018, unknown): “[15] compute local features at different scales for each triangle and arrange them into a rectangular image, which is fed forward through a convolutional neural network (CNN) to predict the semantic label for each triangle.”
Exposure (2017, methodology): “h-level semantic information, from which contextdependent edits can be learned. Deep CNNs have led to clear improvements in a wide range of computer graphics applications, including 3D mesh labeling [Guo et al. 2016] and locomotion modeling [Peng et al. 2016]. For photo retouching, CNNs have been ut”
Deep Marching Cubes: Learning Explicit Surface Representations (2018, background): “Several works consider the problem of shape classification and segmentation by defining neural networks which operate on the graph spanned by the edges and vertices of a 3D mesh [3, 14, 43].”