Multi-directional geodesic neural networks via equivariant convolution
By Adrien Poulenard, M. Ovsjanikov. 2018
103 citations recorded by the source corpus.
We propose a novel approach for performing convolution of signals on curved surfaces and show its utility in a variety of geometric deep learning applications. Key to our construction is the notion of directional functions defined on the surface, which extend the classic real-valued signals and which can be naturally convolved with with real-valued template functions. As a result, rather than trying to fix a canonical orientation or only keeping the maximal response across all alignments of a 2D template at every point of the surface, as done in previous works, we show how information across all rotations can be kept across different layers of the neural network. Our construction, which we call multi-directional geodesic convolution, or directional convolution for short, allows, in particular, to propagate and relate directional information across layers and thus different regions on the shape. We first define directional convolution in the continuous setting, prove its key properties and then show how it can be implemented in practice, for shapes represented as triangle meshes. We evaluate directional convolution in a wide variety of learning scenarios ranging from classification of signals on surfaces, to shape segmentation and shape matching, where we show a significant improvement over several baselines.
Subfields: Computational Mechanics, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computational Mechanics
- M. Ovsjanikov
- Maurice Weiler
- Daniel Cohen-Or
- Keenan Crane
- Souhaib Attaiki
- Nicholas Sharp
- Taco Cohen
- Noa Fish
In Computer Vision and Pattern Recognition
- Maurice Weiler
- Max Welling
- Erik J. Bekkers
- Taco Cohen
- Luc Van Gool
- Daniel Cohen-Or
- Ming-Hsuan Yang
- Andrew Zisserman
Citations in the corpus
- General E(2)-Equivariant Steerable CNNs (2019, neutral): “Our results can therefore readily be transferred to e.g. spherical CNNs [19, 5, 20, 21, 22, 23] or more general models of geometric deep learning [24, 25, 26, 27].”
- Gauge Equivariant Convolutional Networks and the Icosahedral CNN (2019, background): “(Poulenard & Ovsjanikov, 2018) define a multi-directional convolution for “directional functions” (somewhat similar to what we call regular fields), but they parameterize the kernel by a scalar function on the tangent space, which is very limited compared to our matrix-valued kernel (which is the…”
- MeshCNN: a network with an edge (2019, background): “Recently, Li et al. [2018] presented PointCNN, which extends the notion of convolution from a local grid to an Ξ -convolution on a local set of points.”
- DiffusionNet: Discretization Agnostic Learning on Surfaces (2020, methodology): “A particularly challenging setting is extending the power of CNNs to learning directly on curved surfaces [Bronstein et al. 2017; Hanocka et al. 2019; Masci et al. 2015; Poulenard and Ovsjanikov 2018].”