graph2vec: Learning Distributed Representations of Graphs

By Annamalai Narayanan, Mahinthan Chandramohan, R. Venkatesan, Lihui Chen, Yang Liu, Shantanu Jaiswal. 2017

879 citations recorded by the source corpus.

Recent works on representation learning for graph structured data predominantly focus on learning distributed representations of graph substructures such as nodes and subgraphs. However, many graph analytics tasks such as graph classification and clustering require representing entire graphs as fixed length feature vectors. While the aforementioned approaches are naturally unequipped to learn such representations, graph kernels remain as the most effective way of obtaining them. However, these graph kernels use handcrafted features (e.g., shortest paths, graphlets, etc.) and hence are hampered by problems such as poor generalization. To address this limitation, in this work, we propose a neural embedding framework named graph2vec to learn data-driven distributed representations of arbitrary sized graphs. graph2vec's embeddings are learnt in an unsupervised manner and are task agnostic. Hence, they could be used for any downstream task such as graph classification, clustering and even seeding supervised representation learning approaches. Our experiments on several benchmark and large real-world datasets show that graph2vec achieves significant improvements in classification and clustering accuracies over substructure representation learning approaches and are competitive with state-of-the-art graph kernels.

Subfields: Artificial Intelligence, Management Science and Operations Research, Statistical and Nonlinear Physics, Representation & Self-Supervised Learning, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Statistical and Nonlinear Physics

In Representation & Self-Supervised Learning

In Graph Learning

Citations in the corpus

NAS-Bench-NLP: Neural Architecture Search Benchmark for Natural Language Processing (2020, methodology): “Parameterization of architectures We used graph2vec [33] method to create the characteristic features for architectures, which can be useful for some NAS methods like Bayesian Optimization [13].”
Relational inductive biases, deep learning, and graph networks (2018, background): “…which use distributed, vector representations to capture rich semantic content in text (Mikolov et al., 2013; Pennington et al., 2014), graphs (Narayanan et al., 2016, 2017), algebraic and logical expressions (Allamanis et al., 2017; Evans et al., 2018), and programs (Devlin et al., 2017;…”
Unsupervised Inductive Whole-Graph Embedding by Preserving Graph Proximity (2019, methodology): “Thus, we feed the graph-level embeddings generated by UGRAPHEMB and the baselines into a logistic regression classifier to evaluate the quality: (1) GRAPH KERNELS (GK, DGK, SP, DSP, WL, and DWL); (2) GRAPH2VEC (Narayanan et al., 2017); (3) NETMF (Qiu et al.”
Unsupervised Inductive Graph-Level Representation Learning via Graph-Graph Proximity (2019, methodology): “…to evaluate the quality: (1) GRAPH KERNELS (GRAPHLET (GK), DEEP GRAPHLET (DGK), SHORTEST PATH (SP), DEEP SHORTEST PATH (DSP), WEISFEILER-LEHMAN (WL), and DEEP WEISFEILERLEHMAN (DWL)) ; (2) GRAPH2VEC [Narayanan et al., 2017]; (3) NETMF [Qiu et al., 2018]; (4) GRAPHSAGE [Hamilton et al., 2017].”
Capsule Graph Neural Network (2018, methodology): “Deep-Learning-based Methods: Three types of deep-learning-based algorithms are selected: 1) Graph2vec (Narayanan et al., 2017), Deep Graph Kernel (DGK)(Yanardag & Vishwanathan, 2015) and AWE (Ivanov & Burnaev, 2018).”
GraphMAE: Self-Supervised Masked Graph Autoencoders (2022, methodology): “In addition to classical graph kernel methods—WeisfeilerLehman sub-tree kernel (WL) [32] and deep graph kernel (DGK) [47], we also compare GraphMAE with SOTA unsupervised and contrastive methods, GCC [29], graph2vec [25], Infograph [34], GraphCL [54], JOAO [53], MVGRL [11], and InfoGCL [44].”
Graph Contrastive Learning Automated (2021, background): “…& recover, namely AttrMasking (Hu et al., 2019), (3) sub-structure information preserving such as sub2vec (Ad-hikari et al., 2018), graph2vec (Narayanan et al., 2017) and ContextPred (Hu et al., 2019), and (4) global-local representation consistency such as Infomax (Veli ˇ ckovi ´ c et al.,…”
SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation (2022, methodology): “Also, we compare SimGRACE with other graph selfsupervised learning methods: GAE [21], node2vec [12], sub2vec [1], graph2vec [28], EdgePred [16], AttrMasking [16], ContextPred [16], Infomax (DGI) [45], InfoGraph [39] and instance-instance contrastive methods GraphCL [54], JOAO(v2) [53].”
Graph Contrastive Learning with Augmentations (2020, methodology): “Furthermore, GraphCL is evaluated in the unsupervised representation learning following [65, 21], where unsupervised methods generate graph embeddings that are fed into a down-stream SVM classiﬁer [21].”
Contrastive Multi-View Representation Learning on Graphs (2020, methodology): “Finally, We compare the results with ﬁve unsupervised methods including random walk (G¨artner et al., 2003), node2vec (Grover & Leskovec, 2016), sub2vec (Adhikari et al., 2018), graph2vec (Narayanan et al., 2017), and In-foGraph.”
Unsupervised Semantic Segmentation by Distilling Feature Correspondences (2022, background): “This negative shift also appears in the word2vec and graph2vec embedding techniques (Narayanan et al., 2017; Levy & Goldberg, 2014).”
Adversarial Graph Augmentation to Improve Graph Contrastive Learning (2021, background): “Previous works [18,24] show that they generally outperform graph kernels [75–77] and network embedding methods [33,34,78,79].”
UNICORN: Runtime Provenance-Based Detector for Advanced Persistent Threats (2020, background): “By considering such non-linear substructures, in addition to the attributes of each vertex/edge, U NICORN preserves structural equivalence of provenance graphs, which has been demonstrated to outperform linear approaches such as random walk [94].”
A Comprehensive Survey on Graph Anomaly Detection With Deep Learning (2021, methodology): “First, graphs in the database are encoded into a shared latent space using graph-level representation techniques, such as Graph2Vec [144], FGSD [145].”
InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization (2019, methodology): “Recently, however, researchers have proposed algorithms that learn graph-level representations in an unsupervised manner [36, 1].”
You Are What You Do: Hunting Stealthy Malware via Data Provenance Analysis (2020, methodology): “2) Comparison with Different Embedding Approaches: We compare our embedding approach (§V-D) with a graph embedding approach (graph2vec [51]), and the simple nodelevel path embedding (Path Nodes Averaging).”
Self-Supervised Learning of Graph Neural Networks: A Unified Review (2021, background): “It has also achieved remarkable progress in representation learning for different data types, including language sequences [7], [8], [9], images [10], [11], [12], [13], and graphs with sequence models [14], [15] or spectral models [16].”
Graph Learning: A Survey (2021, background): “[98] proposed graph2vec, which is a neural embedding framework.”
Geometric deep learning for computational mechanics Part I: Anisotropic Hyperelasticity (2020, background): “There has been multiple studies on encoding graph structures, spanning from the level of nodes (Grover and Leskovec, 2016) up to the level of entire graphs (Perozzi, Al-Rfou, and Skiena, 2014; Narayanan, Chandramohan, Venkatesan, Chen, Liu, and Jaiswal, 2017).”