InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization

By Fan-Yun Sun, Jordan Hoffmann, Jian Tang. International Conference on Learning Representations, 2019

1,034 citations recorded by the source corpus.

This paper studies learning the representations of whole graphs in both unsupervised and semi-supervised scenarios. Graph-level representations are critical in a variety of real-world applications such as predicting the properties of molecules and community analysis in social networks. Traditional graph kernel based methods are simple, yet effective for obtaining fixed-length representations for graphs but they suffer from poor generalization due to hand-crafted designs. There are also some recent methods based on language models (e.g. graph2vec) but they tend to only consider certain substructures (e.g. subtrees) as graph representatives. Inspired by recent progress of unsupervised representation learning, in this paper we proposed a novel method called InfoGraph for learning graph-level representations. We maximize the mutual information between the graph-level representation and the representations of substructures of different scales (e.g., nodes, edges, triangles). By doing so, the graph-level representations encode aspects of the data that are shared across different scales of substructures. Furthermore, we further propose InfoGraph*, an extension of InfoGraph for semi-supervised scenarios. InfoGraph* maximizes the mutual information between unsupervised graph representations learned by InfoGraph and the representations learned by existing supervised methods. As a result, the supervised encoder learns from unlabeled data while preserving the latent semantic space favored by the current supervised task. Experimental results on the tasks of graph classification and molecular property prediction show that InfoGraph is superior to state-of-the-art baselines and InfoGraph* can achieve performance competitive with state-of-the-art semi-supervised models.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Materials Chemistry

View Paper: https://www.semanticscholar.org/paper/2fb59ebe271d6b007bb0429c1701fd1004782d1b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Materials Chemistry

Citations in the corpus

Towards Domain-Agnostic Contrastive Learning (2020, methodology): “We closely follow the experimental setup of InfoGraph (Sun et al., 2020) for a fair comparison, except that we report results for a linear classifier instead of the Support Vector Classifier applied to the pre-trained representations.”
Graph Neural Networks: A Review of Methods and Applications (2018, background): “Infograph (Sun et al., 2020) aims to learn graph representations by mutual information maximization between graph-level representations and the substructure-level representations of different scales including nodes, edges and triangles.”
Contrastive Multi-View Representation Learning on Graphs (2020, methodology): “…our approach achieves 86.8% accuracy, which is a 5.5% relative improvement over previous state-of-the-art (Velikovi et al., 2019), and on Reddit-Binary graph classiﬁcation benchmark, it achieves 84.5% accuracy, i.e., a 2.4% relative improvement over previous state-of-the-art (Sun et al., 2020).”
Self-Supervised Learning on Graphs: Contrastive, Generative, or Predictive (2021, methodology): “Representative works include DGI [9], InfoGraph [49], HDGI [31], STDGI [28], etc.”
Graph Representation Learning (2020, methodology): “The loss function used in DGI (Equation 6.4) is just one example of a broader class of unsupervised objectives that have witnessed success in the context of GNNs [Hu et al., 2019, Sun et al., 2020].”
SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation (2022, methodology): “Initially, DGI [45] and InfoGraph [37] are proposed to obtain expressive representations for graphs or nodes via maximizing the mutual information between graph-level representations and substructure-level representations of different granularity.”
Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learning (2022, background): “As for graph contrastive learning, existing studies can be categorized into node-level contrastive learning [29, 45] and graph-level contrastive learning [26, 41].”
GPT-GNN: Generative Pre-Training of Graph Neural Networks (2020, background): “Sun et al. present InfoGraph [29], which maximizes the mutual information between graph-level representations obtained from GNNs and the representations of sub-structures.”
Self-Supervised Representation Learning: Introduction, advances, and challenges (2021, neutral): “Several recent techniques optimize mutual information-based, instance discrimination objectives, with DGI [15] and InfoGraph [90] performing contrastive-instance discrimination between pairs of nodes/patches and whole graphs.”
Improving Conversational Recommender Systems via Knowledge Graph based Semantic Fusion (2020, methodology): “y two embedding matrices V (vw for word w) and N ne for item e), respectively. In order to bridge the semantic gap between words and items, we propose to use Mutual Information Maximization technique [25, 28, 33], called MIM. MIM has been used to mutually improve the data representations of two coup”
Graph Clustering with Graph Neural Networks (2020, background): “InfoGraph [55] extended the idea to learning representations of whole graphs instead of just nodes.”
Contrastive Representation Learning: A Framework and Review (2020, neutral): “InfoGraph uses GIN [115] as the base encoder and uses sum over mean for the readout function, both of which are more suitable to learning representations at graph-level.”
Adversarial Graph Augmentation to Improve Graph Contrastive Learning (2021, methodology): “We compare AD-GCL with three unsupervised/self-supervised learning baselines for graph-level tasks, which include randomly initialized untrained GIN (RU-GIN) [72], InfoGraph [18] and GraphCL [24].”
Contrastive Learning with Hard Negative Samples (2020, methodology): “We use the state-of-the-art InfoGraph method introduced by Sun et al. (2020) as the baseline, which is suitable for downstream graph-level classiﬁcation.”
GraphMAE: Self-Supervised Masked Graph Autoencoders (2022, methodology): “In addition to classical graph kernel methods—WeisfeilerLehman sub-tree kernel (WL) [32] and deep graph kernel (DGK) [47], we also compare GraphMAE with SOTA unsupervised and contrastive methods, GCC [29], graph2vec [25], Infograph [34], GraphCL [54], JOAO [53], MVGRL [11], and InfoGCL [44].”
Pre-training Molecular Graph Representation with 3D Geometry (2021, methodology): “Model 2D Topology 3D Geometry EdgePred [31] X AttrMask [38] X GPT-GNN [39] X InfoGraph [71, 79] X ContexPred [38] X GraphLoG [88] X GraphCL [91] X JOAO [90] X Grover [66] X GraphMVP (Ours) X X”
Deep Isolation Forest for Anomaly Detection (2022, methodology): “2) Graph Data and Time Series: As reported in Section VI-A3, traditional isolation-based methods are performed upon vectorised representations learned by unsupervised representation learning method InfoGraph [42] and TS2Vec [43].”
Graph Self-Supervised Learning: A Survey (2021, background): “InfoGraph [Sun et al., 2019] extends such an idea to graph-level tasks, which maximizes the MI between graph representations and substructures at different levels.”