Contrastive Multi-View Representation Learning on Graphs

By Kaveh Hassani, Amir Hosein Khas Ahmadi. 2020

1,725 citations recorded by the source corpus.

We introduce a self-supervised approach for learning node and graph level representations by contrasting structural views of graphs. We show that unlike visual representation learning, increasing the number of views to more than two or contrasting multi-scale encodings do not improve performance, and the best performance is achieved by contrasting encodings from first-order neighbors and a graph diffusion. We achieve new state-of-the-art results in self-supervised learning on 8 out of 8 node and graph classification benchmarks under the linear evaluation protocol. For example, on Cora (node) and Reddit-Binary (graph) classification benchmarks, we achieve 86.8% and 84.5% accuracy, which are 5.5% and 2.4% relative improvements over previous state-of-the-art. When compared to supervised baselines, our approach outperforms them in 4 out of 8 benchmarks. Source code is released at: this https URL

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In Graph Learning

Citations in the corpus

Improving Text-to-Image Synthesis Using Contrastive Learning (2021, background): “In the last two years, various contrastive methods [28, 2, 9, 3, 16, 35, 26, 5, 8, 10, 15, 21, 34, 39] for visual representations have been proposed.”
Self-Supervised Learning: Generative or Contrastive (2020, background): “CMC-Graph [46] Graph C ✓ End-to-end S(2)GRL [90] Graph C End-to-end”
Self-supervised Graph Learning for Recommendation (2020, methodology): “[14] adopt the contrastive model for learning both node and graph representation, which contrasts node representations from one view with graph representation of another view.”
Pre-training Molecular Graph Representation with 3D Geometry (2021, neutral): “[35] Kaveh Hassani and Amir Hosein Khasahmadi.”
Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning (2021, methodology): “Its utility has been validated by other contrastive self-supervised learning works [25], [55], [56].”
Self-Supervised Hypergraph Convolutional Networks for Session-based Recommendation (2020, background): “The dominant paradigm based on cone trastive learning (Hassani and Khasahmadi 2020; Qiu et al. 2020a) suggests that contrasting congruent and incongruent views of graphs with mutual information maximization can help encode rich graph/node representations.”
SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation (2022, methodology): “More recently, MVGRL [13] proposes to learn both node-level and graph-level representation by performing node diffusion and contrasting node representation to augmented graph representations.”
Multi-view Contrastive Graph Clustering (2021, positive): “On the other hand, contrastive multi-view representation learning on graphs (MVGRL) [Hassani and Khasahmadi, 2020] method is proposed, which performs representation learning by contrasting two diffusion matrices transformed from the adjacency matrix.”
Self-Supervised Learning of Graph Neural Networks: A Unified Review (2021, methodology): “For example, Sun et al. [41], Hassani and Khasahmadi [10] use 3-layer MLPs, You et al. [49] use 2-layer MLPs as the projection heads and Sun et al. [41] applies a linear projection to the graph-level representation.”
Protein Representation Learning by Geometric Structure Pretraining (2022, background): “…Hamilton et al., 2017), context prediction (Hu et al., 2019; Rong et al., 2020), node/edge attribute reconstruction (Hu et al., 2019) and contrastive learning (Hassani & Khasahmadi, 2020; Qiu et al., 2020; You et al., 2020; Xu et al., 2021), are designed to acquire knowledge from unlabeled graphs.”
Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning (2021, background): “Contrastive learning, as one typical technique of self-supervised learning, has attracted considerable attentions [1, 11, 12, 30].”
Deep multi-view learning methods: A review (2021, background): “[76] present a contrastive multi-view feature learning method on graphs, which indicates that increasing the number of views to more than two cannot improve performance.”
Graph Contrastive Learning with Augmentations (2020, background): “…with various choices of the compositions result in (4) instantiating as other speciﬁc contrastive learning algorithms including [54, 55, 56, 21, 57, 58, 59] also shown in in Appendix F. 4 The Role of Data Augmentation in Graph Contrastive Learning In this section, we assess and rationalize the…”
Bootstrapped Representation Learning on Graphs (2021, methodology): “Ad-ditionally, MVGRL method (Hassani & Khasahmadi, 2020) generalizes CMC (Tian et al., 2020) to graphs.”
Adversarial Graph Augmentation to Improve Graph Contrastive Learning (2021, background): “To avoid the above issue, graph contrastive learning (GCL) has attracted more attention recently [18, 22,23,25–31].”
Graph Contrastive Learning with Adaptive Augmentation (2020, methodology): “Following DGI, GMI [32] employs two discriminators to directly measure MI between input and representations of both nodes and edges without data augmentation; MVGRL [16] proposes to learn both node- and graph-level representations by performing node diffusion and contrasting node representations to ”
Large-Scale Representation Learning on Graphs via Bootstrapping (2021, positive): “…commonly used self-supervised graph methods from previously published results (Perozzi et al., 2014; Veliˇckovi´c et al., 2019; Peng et al., 2020; Hassani & Khasahmadi, 2020; Zhu et al., 2020a), as well as Random-Init (Veliˇckovi´c et al., 2019), a baseline using embeddings from a randomly…”