Deep Functional Maps: Structured Prediction for Dense Shape Correspondence

By Or Litany, Tal Remez, Emanuele Rodolà, A. Bronstein, Michael M. Bronstein. 2017

346 citations recorded by the source corpus.

We introduce a new framework for learning dense correspondence between deformable 3D shapes. Existing learning based approaches model shape correspondence as a labelling problem, where each point of a query shape receives a label identifying a point on some reference domain; the correspondence is then constructed a posteriori by composing the label predictions of two input shapes. We propose a paradigm shift and design a structured prediction model in the space of functional maps, linear operators that provide a compact representation of the correspondence. We model the learning process via a deep residual network which takes dense descriptor fields defined on two shapes as input, and outputs a soft map between the two given objects. The resulting correspondence is shown to be accurate on several challenging benchmarks comprising multiple categories, synthetic models, real scans with acquisition artifacts, topological noise, and partiality.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Mathematical Optimization

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

In Mathematical Optimization

Citations in the corpus

Geometric Deep Learning: Going beyond Euclidean data (2016, background): “[109] proposed intrinsic structured prediction of shape correspondence by integrating a layer computing functional correspondence [106] into the deep neural network.”
Dynamic Graph CNN for Learning on Point Clouds (2018, background): “In [Halimi et al. 2018; Litany et al. 2017b], a differentiable functional map [Ovsjanikov et al.”
SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels (2017, methodology): “While we train against a one-hot binary vector using the cross entropy loss, FMNet trains using a specialized soft error loss, which is a more geometrically meaningful criterion that punishes geodesically far-away predictions stronger than predictions near the correct node [16].”
3D shape segmentation via shape fully convolutional networks (2017, neutral): “re generalized CNN architectures by using mixture model. [35] using a global seamless parameterization for which the convolution operator is well deﬁned but can only be used on sphere-like meshes. As [36, 37] show that intrinsic CNN methods can achieve better result on shape correspondence and descr”
Deformable Shape Completion with Graph Convolutional Autoencoders (2017, background): “Dense non-rigid shape correspondence [23, 11, 33, 47, 8] is a fundamental challenge as it is an enabler for many high level tasks like pose or texture transfer across surfaces.”
Image Matching from Handcrafted to Deep Features: A Survey (2020, methodology): “This approachwas adopted and extended inmany follow-upworks (Aflalo et al. 2016;Kovnatsky et al. 2015; Pokrass et al. 2013; Rodolà et al. 2017; Litany et al. 2017).”
DiffusionNet: Discretization Agnostic Learning on Surfaces (2020, methodology): “However, the features are still learned either with MLPs starting from pre-computed descriptors [Ginzburg and Raviv 2020; Halimi et al. 2019; Litany et al. 2017; Roufosse et al. 2019] or using point-based architectures [Donati et al. 2020].”
On Learning Sets of Symmetric Elements (2020, methodology): “When this correspondence is not available, a shape matching al-gorithm like (Litany et al., 2017; Maron & Lipman, 2018) can be used as preprocessing.”
3D-CODED: 3D Correspondences by Deep Deformation (2018, methodology): “Ours FMNet [24] GCNN [28] LSCNN [8] ADD3 [9] Results on SCAPE : real and partial data.”
PRNet: Self-Supervised Learning for Partial-to-Partial Registration (2019, methodology): “Mathematical methods like functional maps can be made ‘deep’ using priors learned from data: Deep functional maps [43, 44] learn descriptors rather than designing them by hand.”