Surface Networks

By Ilya Kostrikov, Joan Bruna, Daniele Panozzo, Denis Zorin. 2017

108 citations recorded by the source corpus.

We study data-driven representations for three-dimensional triangle meshes, which are one of the prevalent objects used to represent 3D geometry. Recent works have developed models that exploit the intrinsic geometry of manifolds and graphs, namely the Graph Neural Networks (GNNs) and its spectral variants, which learn from the local metric tensor via the Laplacian operator. Despite offering excellent sample complexity and built-in invariances, intrinsic geometry alone is invariant to isometric deformations, making it unsuitable for many applications. To overcome this limitation, we propose several upgrades to GNNs to leverage extrinsic differential geometry properties of three-dimensional surfaces, increasing its modeling power. In particular, we propose to exploit the Dirac operator, whose spectrum detects principal curvature directions - this is in stark contrast with the classical Laplace operator, which directly measures mean curvature. We coin the resulting models Surface Networks (SN). We prove that these models define shape representations that are stable to deformation and to discretization, and we demonstrate the efficiency and versatility of SNs on two challenging tasks: temporal prediction of mesh deformations under non-linear dynamics and generative models using a variational autoencoder framework with encoders/decoders given by SNs.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

Dynamic Graph CNN for Learning on Point Clouds (2018, background): “Kostrikov et al. [2017] proposed SurfaceNets based on the extrinsic Dirac operator for this task.”
Deformable Shape Completion with Graph Convolutional Autoencoders (2017, methodology): “In [52], a VAE for non-rigid shapes is proposed.”
MeshCNN: a network with an edge (2019, unknown): “One approach advocates for the processing of the Laplacian of the graph representation [Bruna et al. 2014; Henaff et al. 2015; Defferrard et al. 2016; Kostrikov et al. 2018], and thus operates in the spectral domain.”
Multi-directional geodesic neural networks via equivariant convolution (2018, background): “Finally, a very recent Surface Networks approach [Kostrikov et al. 2017] is based on stacking layers consiting of combinations of features and their images by the Laplace or Dirac operator to exploit intrinsic and extrinsic information.”
DiffusionNet: Discretization Agnostic Learning on Surfaces (2020, methodology): “We also note that our use of the Laplacian in defining the diffusion operator is related to methods based on polynomials of the Laplacian [Defferrard et al. 2016; Kostrikov et al. 2018], CayleyNets [Levie et al. 2018] and their recent application in shape matching using ACSCNNs [Li et al. 2020b].”
ABC: A Big CAD Model Dataset for Geometric Deep Learning (2018, methodology): “Surface Networks [36] further proposes the use of the differential geometry operators to extend GNNs to exploit properties of the surface.”