Spherical CNNs on Unstructured Grids

By Chiyu “Max”, Jiang, Jingwei Huang, Karthik Kashinath, Lawrence Berkeley Nat’l Lab, P. Marcus, Matthias Nießner, MeshConv MeshConv. 2019

204 citations recorded by the source corpus.

We present an efficient convolution kernel for Convolutional Neural Networks (CNNs) on unstructured grids using parameterized differential operators while focusing on spherical signals such as panorama images or planetary signals. To this end, we replace conventional convolution kernels with linear combinations of differential operators that are weighted by learnable parameters. Differential operators can be efficiently estimated on unstructured grids using one-ring neighbors, and learnable parameters can be optimized through standard back-propagation. As a result, we obtain extremely efficient neural networks that match or outperform state-of-the-art network architectures in terms of performance but with a significantly lower number of network parameters. We evaluate our algorithm in an extensive series of experiments on a variety of computer vision and climate science tasks, including shape classification, climate pattern segmentation, and omnidirectional image semantic segmentation. Overall, we present (1) a novel CNN approach on unstructured grids using parameterized differential operators for spherical signals, and (2) we show that our unique kernel parameterization allows our model to achieve the same or higher accuracy with significantly fewer network parameters.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Environmental Engineering, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Environmental Engineering

In Model Architectures

Citations in the corpus

Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data (2020, methodology): “While the approach has been applied on general data types like spherical images (Esteves et al., 2018; Cohen et al., 2018; Jiang et al., 2019), voxel data (Weiler et al.”
General E(2)-Equivariant Steerable CNNs (2019, background): “The presented solutions of the kernel space constraint therefore describe spherical CNNs [19, 5, 20, 21, 22, 23] or convolutional networks on triangulated meshes [24, 25, 26, 27] for different choices of structure groups and group representations.”
Gauge Equivariant Convolutional Networks and the Icosahedral CNN (2019, positive): “The preprocessed data as released by (Jiang et al., 2018) consists of 16 -channel spherical images at resolution r = 5 , which we reinterpret as icosahedral signals (introducing slight distortion).”
Improving Data‐Driven Global Weather Prediction Using Deep Convolutional Neural Networks on a Cubed Sphere (2020, methodology): “are not inherently accounted for in this method. Other studies have proposed several methods of applying convolutions on icosohedral grids, which promise to have less distortion than the cubed sphere [22, 17, 44]; however, these methods require complex adaptation of existing CNN software libraries a”
WeatherBench: A Benchmark Data Set for Data‐Driven Weather Forecasting (2020, background): “tion of wind caused by the rotation of Earth, which also depends on latitude. A possible solution in the horizontal could be to use spherical convolutions (Cohen et al., 2018; Perraudin et al., 2019; Jiang et al., 2019) or to feed in latitude information to the network. Another potential issue is th”
Local Implicit Grid Representations for 3D Scenes (2020, methodology): “[27, 17, 20, 19] have been proposed for analyzing signals on meshes, and [37, 15, 7] for generating mesh representations.”
Physics-informed machine learning: case studies for weather and climate modelling (2021, methodology): “Jiang et al. [29] extended this to unstructured grids using parametric differential operators to develop efﬁcient and compact networks with high accuracy on climate pattern segmentation.”
DiffusionNet: Discretization Agnostic Learning on Surfaces (2020, methodology): “These gradient features are built on local differential operators, which have also been exploited in other recent methods ( e.g. , [Eliasof and Treister 2020; Jiang et al. 2018]).”
Geometric Structure Based and Regularized Depth Estimation From 360 Indoor Imagery (2020, background): “Very recently, more efﬁcient re-projection based approaches have been proposed, including distortion-aware convolution [3, 9, 31, 37], spherical convolutions with crown kernels [36], and spherical convolu-tions operating on the PHD [22] or unstructured grids [17].”
Shape As Points: A Differentiable Poisson Solver (2021, background): “Explicit shape representations and learning algorithms depending on such representations directly parameterize the surface of the geometry, either as a point cloud [17, 40, 55, 56, 67, 70], parameterized mesh [22, 26, 29, 64] or surface patches [2,20,39,45,68,71,72].”