Learning class‐specific descriptors for deformable shapes using localized spectral convolutional networks

By Davide Boscaini, Jonathan Masci, Simone Melzi, Michael M. Bronstein, Umberto Castellani, Pierre Vandergheynst. 2015

230 citations recorded by the source corpus.

In this paper, we propose a generalization of convolutional neural networks (CNN) to non‐Euclidean domains for the analysis of deformable shapes. Our construction is based on localized frequency analysis (a generalization of the windowed Fourier transform to manifolds) that is used to extract the local behavior of some dense intrinsic descriptor, roughly acting as an analogy to patches in images. The resulting local frequency representations are then passed through a bank of filters whose coefficient are determined by a learning procedure minimizing a task‐specific cost. Our approach generalizes several previous methods such as HKS, WKS, spectral CNN, and GPS embeddings. Experimental results show that the proposed approach allows learning class‐specific shape descriptors significantly outperforming recent state‐of‐the‐art methods on standard benchmarks.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs (2016, background): “In [6], the same authors proposed a CNN-type architecture in the spatio-frequency domain us-ing the windowed Fourier transform formalism [48].”
General E(2)-Equivariant Steerable CNNs (2019, background): “The presented solutions of the kernel space constraint therefore describe spherical CNNs [19, 5, 20, 21, 22, 23] or convolutional networks on triangulated meshes [24, 25, 26, 27] for different choices of structure groups and group representations.”
Gauge Equivariant Convolutional Networks and the Icosahedral CNN (2019, methodology): “Finally, we consider a model that uses only rotation-invariant ﬁlters, i.e. scalar-to-scalar (S2S) convolutions, similar to standard graph CNNs (Boscaini et al., 2015; Kipf & Welling, 2017).”
Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks (2017, background): “…deep learning (Bron-stein et al., 2016) techniques that can mathematically deal with graph-structured data, which arises in numerous applications, ranging from computer graphics and vision (Masci et al., 2015; Boscaini et al., 2015; 2016b;a; Monti et al., 2017) to chemistry (Duvenaud et al., 2015).”
Geometric Deep Learning: Going beyond Euclidean data (2016, background): “An additional degree of freedom is the deﬁnition of the window, which can also be learned [27].”
Deep Learning on Lie Groups for Skeleton-Based Action Recognition (2016, background): “Moreover, recently some deep learning models have emerged [10, 7, 27, 24, 18] that deal with data in a non-Euclidean domain.”
SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation (2016, background): “There have been some previous works studying graph CNN on more general graphs instead of 2D regular grids [3, 6, 8, 5], and [19, 1, 2] have a special focus on 3D shape graphs like human bodies.”
Generating 3D faces using Convolutional Mesh Autoencoders (2018, methodology): “[4] extend this using a windowed Fourier transform to localize in frequency space.”
Learning shape correspondence with anisotropic convolutional neural networks (2016, methodology): “Yet, unlike the geodesic patches used in GCNN, the disadvantage of LSCNN is the lack of oriented structures which tend to be important in capturing the local context.”
FPNN: Field Probing Neural Networks for 3D Data (2016, background): “While 3D data is represented as meshes, neural networks can benefit from intrinsic constructions[17, 18, 1, 2] to learn object invariance to isometries, thus require less training data for handling deformable objects.”
Deep Functional Maps: Structured Prediction for Dense Shape Correspondence (2017, methodology): “We compare with state-of-the-art learning-based approaches for deformable shape correspondence, namely optimal spectral descriptors (OSD) [28], geodesic CNNs (GCNN) [29], localized spectral CNNs (LSCNN) [8], and two variants of anisotropic diffusion descriptors (ADD3, mADD3) [10].”
Dynamic FAUST: Registering Human Bodies in Motion (2017, background): “From this dataset, researchers have used both real scans (for benchmarking registration techniques [20, 51, 55]) and aligned templates (e.g. for training Convolutional Neural Networks [9, 10, 11, 33, 53]).”
Geodesic Convolutional Neural Networks on Riemannian Manifolds (2015, methodology): “Intrinsic versions of CNNs that would allows dealing with shape deformations are difficult to formulate due to the lack of shift invariance on Riemannian manifolds; we are aware of two recent works in that direction [9, 5].”
A Riemannian Network for SPD Matrix Learning (2016, methodology): “Following (Bruna et al. 2014), a localized spectral network was proposed in (Boscaini et al. 2015) to non-Euclidean domains by generalizing the windowed Fourier transform to manifolds to extract the local behavior of some dense intrinsic descriptor.”
3D2SeqViews: Aggregating Sequential Views for 3D Global Feature Learning by CNN With Hierarchical Attention Aggregation (2019, background): “[14] proposed localized spectral convolutional networks to perform supervised local feature learning.”
Graph-Based Classification of Omnidirectional Images (2017, methodology): “Luckily graph-based deep learning techniques have been recently introduced [13, 14, 15] that allow applying deep learning techniques to irregularly structured data.”