Learning shape correspondence with anisotropic convolutional neural networks

By Davide Boscaini, Jonathan Masci, Emanuele Rodolà, Michael M. Bronstein. 2016

537 citations recorded by the source corpus.

Establishing correspondence between shapes is a fundamental problem in geometry processing, arising in a wide variety of applications. The problem is especially difficult in the setting of non-isometric deformations, as well as in the presence of topological noise and missing parts, mainly due to the limited capability to model such deformations axiomatically. Several recent works showed that invariance to complex shape transformations can be learned from examples. In this paper, we introduce an intrinsic convolutional neural network architecture based on anisotropic diffusion kernels, which we term Anisotropic Convolutional Neural Network (ACNN). In our construction, we generalize convolutions to non-Euclidean domains by constructing a set of oriented anisotropic diffusion kernels, creating in this way a local intrinsic polar representation of the data (`patch'), which is then correlated with a filter. Several cascades of such filters, linear, and non-linear operators are stacked to form a deep neural network whose parameters are learned by minimizing a task-specific cost. We use ACNNs to effectively learn intrinsic dense correspondences between deformable shapes in very challenging settings, achieving state-of-the-art results on some of the most difficult recent correspondence benchmarks.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs (2016, methodology): “Both GCNN and ACNN operate in the spatial domain and thus do not suffer from the inherent inability of spectral methods to generalize across different domains.”
Gauge Equivariant Convolutional Networks and the Icosahedral CNN (2019, background): “Hence, this general theory can describe in a uniﬁed way the most prominent and geometrically natural methods of geometrical deep learning (Masci et al., 2015; Boscaini et al., 2016), as well as all G-CNNs on homogeneous spaces.”
Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks (2017, background): “New neural networks are necessary and this has motivated the recent development of geometric deep learning techniques that can mathematically deal with graph-structured data, which arises in numerous applications, ranging from computer graphics and vision [28, 2, 4, 3, 30] to chemistry [12].”
Geometric Deep Learning: Going beyond Euclidean data (2016, methodology): “The discretization of the anisotropic Laplacian is a modification of the cotangent formula (14) on meshes or graph Laplacian (11) on point clouds [48].”
SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation (2016, methodology): “ACNN [2] does not explicitly conduct multi-scale analysis and is not designed for 3D shapes with large non-isometric variations, thus generalizes less well across a diverse set of shapes.”
Generating 3D faces using Convolutional Mesh Autoencoders (2018, methodology): “This method resembles the intrinsic mesh convolution of [31] and [5] for specific choices of the weight functions.”
Dynamic Graph CNN for Learning on Point Clouds (2018, methodology): “Follow-up work proposed different local charting techniques using anisotropic diffusion [Boscaini et al. 2016] or Gaussian mixture models [Monti et al.”
SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels (2017, methodology): “[18, 2, 17, 16], we go without handcrafted feature descriptors as inputs, like the local histogram of normal vectors known as SHOT descriptors [24], and force the network to learn from the geometry (i.”
PCPNet Learning Local Shape Properties from Raw Point Clouds (2017, methodology): “However, several methods have also been proposed for shape segmentation [GZC15,MGA ∗ 17] and even shape correspondence [MBBV15,WHC ∗ 16,BMRB16], among many others.”
Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images (2018, methodology): “In the meanwhile, there are charting-based methods which directly apply convolutions on surface manifolds [2, 22, 23] for shape analysis.”
A Papier-Mache Approach to Learning 3D Surface Generation (2018, neutral): “The FAUST dataset [4] is a collection of meshes representing several humans in different poses.”
3D shape segmentation via shape fully convolutional networks (2017, neutral): “ better understanding the intrinsic structures, many works directly adapt neural networks to meshes. The ﬁrst mesh CNN method GCNN[32] used convolution ﬁlters in local geodesic polar coordinates. ACNN[33] used anisotropic heat kernels as ﬁlters. MoNet[34] considered a more generalized CNN architectu”
Learning SO(3) Equivariant Representations with Spherical CNNs (2017, background): “There are two threads within this space, spectral (Bruna et al. 2013b; Defferrard et al. 2016; Kipf and Welling 2016; Yi et al. 2016) and spatial (Boscaini et al. 2016; Masci et al. 2015; Monti et al. 2016; Wang et al. 2018).”
VConv-DAE: Deep Volumetric Shape Learning Without Object Labels (2016, background): “Following the success of Shapenet, there have been recent work that improves the recognition results on 3D data [10], uses 3D-2D(multi-view images) correspondence to improve shape completion (repairing) results [11,12] propose intrinsic CNN [13,14] for 3D data or learn correspondence between two sur”
Deep Functional Maps: Structured Prediction for Dense Shape Correspondence (2017, methodology): “[9] and generalized further by Monti et al.”
Spherical CNNs on Unstructured Grids (2019, methodology): “Boscaini et al. (2016) introduce a generalization of classic CNNs to non-Euclidean domains by using a set of oriented anisotropic diffusion kernels.”
Deformable Shape Completion with Graph Convolutional Autoencoders (2017, methodology): “The first intrinsic CNN-type architectures for 3D shapes were based on local charting techniques generalizing the notion of “patches” to nonEuclidean and irregularly-sampled domains [35, 7, 36].”
Multi-directional geodesic neural networks via equivariant convolution (2018, methodology): “Finally, we extend previous approaches such as [Boscaini et al. 2016; Monti et al. 2017] by adding spatial pooling layers through mesh simplification and exploiting residual learning (ResNet) blocks [He et al.”
DiffusionNet: Discretization Agnostic Learning on Surfaces (2020, methodology): “…this challenge through a variety of solutions such as mapping to a canonical domain [Maron et al. 2017; Sinha et al. 2016], exploiting local parametrizations [Boscaini et al. 2016; Masci et al. 2015; Wiersma et al. 2020] or applying convolution on the edges of the mesh [Hanocka et al. 2019].”