Deep Convolutional Networks on Graph-Structured Data

By Mikael Henaff, Joan Bruna, Yann LeCun. 2015

1,674 citations recorded by the source corpus.

Deep Learning's recent successes have mostly relied on Convolutional Networks, which exploit fundamental statistical properties of images, sounds and video data: the local stationarity and multi-scale compositional structure, that allows expressing long range interactions in terms of shorter, localized interactions. However, there exist other important examples, such as text documents or bioinformatic data, that may lack some or all of these strong statistical regularities. In this paper we consider the general question of how to construct deep architectures with small learning complexity on general non-Euclidean domains, which are typically unknown and need to be estimated from the data. In particular, we develop an extension of Spectral Networks which incorporates a Graph Estimation procedure, that we test on large-scale classification problems, matching or improving over Dropout Networks with far less parameters to estimate.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Graph Learning

Citations in the corpus

Graph Attention Networks (2017, background): “Henaff et al. (2015) introduced a parameterization of the spectral ﬁlters with smooth coefﬁcients in order to make them spatially localized.”
Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering (2016, methodology): “Besides, we addressed the three concerns raised by [13]: (i) we introduced a model whose computational complexity is linear with the dimensionality of the data, (ii) we confirmed that the quality of the input graph is of paramount importance, (iii) we showed that the statistical assumptions of local”
Learning Convolutional Neural Networks for Graphs (2016, background): “We provide a brief introduction to the required background in convolutional networks and graph theory.”
A simple neural network module for relational reasoning (2017, background): “approaches [11, 32] and has been explicitly pursued using neural networks as well [4]. There is recent work applying neural networks to graphs, which are a natural structure for formalising relations [12, 19, 33, 37, 26, 2]. Perhaps a crucial dierence between this work and our work here is that RNs ”
Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs (2016, background): “[18] argued that smooth spectral filter coefficients result in spatially-localized filters and used parametric filters of the”
Discriminative Embeddings of Latent Variable Models for Structured Data (2016, neutral): “, 2013), or graph Fourier transform (Henaff et al., 2015).”
On the Binding Problem in Artificial Neural Networks (2020, background): “On the other hand, some GCNs (e.g. Henaff et al., 2015; Lee et al., 2019) have used a mechanism for coarsening (down-sampling) the graph between layers, to reduce computational complexity, which could provide a mechanism for refining the structure (i.e. structure inference).”
Relational inductive biases, deep learning, and graph networks (2018, background): “…neural network (MPNN), which uniﬁed various graph neural network and graph convolutional network approaches (Monti et al., 2017; Bruna et al., 2014; Henaﬀ et al., 2015; Deﬀerrard et al., 2016; Niepert et al., 2016; Kipf and Welling, 2017; Bronstein et al., 2017) by analogy to message-passing in…”
Flexible Neural Representation for Physics Prediction (2018, positive): “Our work draws inspiration from and extends that of Chang et al. [11] and Battaglia et al. [4], which in turn use ideas from graph-based neural networks [39, 44, 9, 30, 22, 14, 13, 24, 8, 40].”
The More You Know: Using Knowledge Graphs for Image Classification (2016, background): “There has been a lot of work in end-to-end learning on graphs or neural network trained on graphs [31, 2, 6, 11, 25, 22, 9, 21].”
An End-to-End Deep Learning Architecture for Graph Classification (2018, background): “…1) given a single graph structure, infer signals in graph forms or infer labels of individual nodes (Scarselli et al. 2009; Bruna et al. 2013; Henaff, Bruna, and LeCun 2015; Li et al. 2015; Defferrard, Bresson, and Vandergheynst 2016; Kipf and Welling 2016); and 2) given a set of graphs with…”
Learning Multimodal Graph-to-Graph Translation for Molecular Optimization (2018, background): “Previous work on graph encoders includes convolutional (Scarselli et al., 2009; Bruna et al., 2013; Henaff et al., 2015; Duvenaud et al., 2015; Niepert et al., 2016; Defferrard et al., 2016; Kondor et al., 2018) and recurrent architectures (Li et al., 2015; Dai et al., 2016; Lei et al., 2017).”
Covariant Compositional Networks For Learning Graphs (2018, background): “This connection has inspired several authors to frame graph learning as a generalization of convolutional nets to the graph domain (Bruna et al., 2014; Henaff et al., 2015; Duvenaud et al., 2015; Defferrard et al., 2016; Kipf & Welling, 2017).”
SchNet: A continuous-filter convolutional neural network for modeling quantum interactions (2017, background): “Therefore, various extensions of convolutional layers even beyond the Euclidean space exist, e.g., for graphs [31, 32] and 3d shapes[33].”
Deriving Neural Architectures from Sequence and Graph Kernels (2017, methodology): “Other approaches, e.g., Bruna et al. (2013) and Henaff et al. (2015), rely on graph Laplacian or Fourier transform.”
Cormorant: Covariant Molecular Neural Networks (2019, background): “Similar ideas underly generalizations of CNNs to manifolds [Masci et al., 2015, Monti et al., 2016, Bronstein et al., 2017] and graphs [Bruna et al., 2014, Henaff et al., 2015].”