Spectral Networks and Locally Connected Networks on Graphs
By Joan Bruna, Wojciech Zaremba, Arthur Szlam, Yann LeCun. 2013
5,411 citations recorded by the source corpus.
Convolutional Neural Networks are extremely efficient architectures in image and audio recognition tasks, thanks to their ability to exploit the local translational invariance of signal classes over their domain. In this paper we consider possible generalizations of CNNs to signals defined on more general domains without the action of a translation group. In particular, we propose two constructions, one based upon a hierarchical clustering of the domain, and another based on the spectrum of the graph Laplacian. We show through experiments that for low-dimensional graphs it is possible to learn convolutional layers with a number of parameters independent of the input size, resulting in efficient deep architectures.
Subfields: Applied Mathematics, Artificial Intelligence, Graph Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Applied Mathematics
- Joan Bruna
- Michael M. Bronstein
- Max Welling
- Pierre Vandergheynst
- Thomas Kipf
- Pascal Frossard
- José M. F. Moura
- William L. Hamilton
In Artificial Intelligence
- Yoshua Bengio
- Mohit Bansal
- Percy Liang
- Quoc V. Le
- Jianfeng Gao
- Sergey Levine
- Ruslan Salakhutdinov
- Dacheng Tao
In Graph Learning
- Jure Leskovec
- William L. Hamilton
- Stephan Günnemann
- Matthias Fey
- Michael M. Bronstein
- Aleksandar Bojchevski
- Rex Ying
- Max Welling
Citations in the corpus
- Semi-Supervised Classification with Graph Convolutional Networks (2016, methodology): “Our method is based on spectral graph convolutional neural networks, introduced in Bruna et al. (2014) and later extended by Defferrard et al. (2016) with fast localized convolutions.”
- Graph Attention Networks (2017, methodology): “In Bruna et al. (2014), the convolution operation is defined in the Fourier domain by computing the eigendecomposition of the graph Laplacian, resulting in potentially intense computations and non-spatially localized filters.”
- Inductive Representation Learning on Large Graphs (2017, neutral): “The majority of these methods do not scale to large graphs or are designed for whole-graph classification (or both) [4, 9, 8, 24].”
- Learning Convolutional Neural Networks for Graphs (2016, background): “Recent work extended CNNs to topologies that differ from the low-dimensional grid structure (Bruna et al., 2014; Henaff et al., 2015).”
- Gated Graph Sequence Neural Networks (2015, methodology): “…this work are methods that learn features on graphs, including Graph Neural Networks (Gori et al., 2005; Scarselli et al., 2009), spectral networks (Bruna et al., 2013) and recent work on learning graph fingerprints for classification tasks on graph representations of chemical molecules (Duvenaud et…”
- Convolutional Networks on Graphs for Learning Molecular Fingerprints (2015, background): “To avoid this problem, [2] proposed a hierarchical clustering of graph substructures.”
- VAIN: Attentional Multi-agent Predictive Modeling (2017, neutral): “Notable works in graph NNs includes Spectral Networks [16] and work by Duvenaud et al.”
- Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs (2016, background): “…in non-Euclidean deep learning has recently surged in the computer vision and machine learning communities after the seminal work of Bruna et al. [12, 22], in which the authors formulated CNN-like [28] deep neural architectures on graphs in the spectral domain, employing the analogy…”
- Deep Convolutional Networks on Graph-Structured Data (2015, methodology): “Whereas some recognition tasks in non-Euclidean domains, such as those considered in [2] or [12], might have a prior knowledge of the graph structure of the input data, many other real-world applications do not have such knowledge.”
- PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation (2016, background): “Spectral CNNs: Some latest works [4, 14] use spectral CNNs on meshes.”
- Discriminative Embeddings of Latent Variable Models for Structured Data (2016, neutral): “There are several work transferring locality concept of convolutional neural networks (CNN) from Euclidean domain to graph case, using hierarchical clustering, graph Laplacian (Bruna et al., 2013), or graph Fourier transform (Henaff et al., 2015).”
- Neural Logic Machines (2019, background): “Graph convolution networks (GCNs) (Bruna et al., 2014; Li et al., 2016; Defferrard et al., 2016; Kipf & Welling, 2017) is a family of neural architectures working on graphs.”
- Knowledge Graph Convolutional Networks for Recommender Systems (2019, methodology): “red by GCN. In general, GCN can be categorized as spectral methods and non-spectral methods. Spectral methods represent graphs and perform convolution in the spectral space. For example, Bruna et al. [2] define the convolution in Fourier domain and calculates the eigendecomposition of the graph Lapl”
- Neural Message Passing for Quantum Chemistry (2017, background): “We closely follow the notation defined in Bruna et al. (2013) equation (3.2).”
- E(n) Equivariant Graph Neural Networks (2021, methodology): “Horie et al. (2020) proposes a set of isometric invariant and equivairant transformations for Graph Neural Networks.”
- Graph Convolutional Neural Networks for Web-Scale Recommender Systems (2018, background): “(2013), which developed a version of graph convolutions based on spectral graph thery [7].”