Multi-scale Attributed Node Embedding

By Benedek Rozemberczki, Carl Allen, Rik Sarkar. J. Complex Networks, 2019

1,111 citations recorded by the source corpus.

We present network embedding algorithms that capture information about a node from the local distribution over node attributes around it, as observed over random walks following an approach similar to Skip-gram. Observations from neighbourhoods of different sizes are either pooled (AE) or encoded distinctly in a multi-scale approach (MUSAE). Capturing attribute-neighbourhood relationships over multiple scales is useful for a range of applications, including latent feature identification across disconnected networks with similar features. We prove theoretically that matrices of node-feature pointwise mutual information are implicitly factorized by the embeddings. Experiments show that our algorithms are computationally efficient and outperform comparable models on social networks and web graphs.

Subfields: Artificial Intelligence, Information Systems, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/135334ea7fdef8eef0367e862797cac7dcd232a4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Statistical and Nonlinear Physics

Citations in the corpus

Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods (2021, background): “[58] (collected by [61, 66, 48]) that have certain undesirable properties such as small size, narrow range of application areas, and high variance between different train/test splits [82].”
Revisiting Heterophily For Graph Neural Networks (2022, methodology): “On each dataset used in [37, 36], we test the models 10 times following the same early stopping strategy, the same 60%/20%/20% random data split 16 and Adam [18] optimizer as used in GPRGNN [8].”
BernNet: Learning Arbitrary Graph Spectral Filters via Bernstein Approximation (2021, methodology): “On Chameleon and Squirrel, BernNet learns two comb-alike filters.”
A critical look at the evaluation of GNNs under heterophily: are we really making progress? (2023, methodology): “These datasets are initially collected by Rozemberczki et al. (2021): nodes represent articles from the English Wikipedia (December 2018), and edges reﬂect mutual links between them.”
Beyond Homophily in Graph Neural Networks: Current Limitations and Effective Designs (2020, methodology): “Real datasets & setup We now evaluate the performance of our model and existing GNNs on a variety of real-world datasets [35, 29, 30, 22, 4, 31] with edge homophily ratio h ranging from strong heterophily to strong homophily, going beyond the traditional Cora, Pubmed and Citeseer graphs that have…”
HGNN+: General Hypergraph Neural Networks (2022, methodology): “Graph Structure 6.2.1 Datasets Five public benchmarks are selected in this experiment, which belong to two categories, i.e., publication citation network (Cora [25], Citeseer and Pubmed [26]) and social media network (Github Web ML and Facebook [27]).”
Simple and Deep Graph Convolutional Networks (2020, methodology): “Following the setting in (Pei et al., 2020), we use 7 datasets: Cora, Citeseer, Pubmed, Chameleon, Method PPI GraphSAGE (Hamilton et al., 2017) 61.2 VR-GCN (Chen et al., 2018b) 97.8 GaAN (Zhang et al., 2018) 98.71 GAT (Veliˇckovi´c et al., 2018) 97.3 JKNet (Xu et al., 2018) 97.6 GeniePath (Liu et al”
Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks (2021, methodology): “We evaluate the performance of our GGCN model and existing GNNs in node classification on various realworld datasets (Tang et al., 2009; Rozemberczki et al., 2019; Sen et al., 2008; Namata et al., 2012; Bojchevski & Günnemann, 2018; Shchur et al., 2018).”
Adaptive Universal Generalized PageRank Graph Neural Network (2020, background): “Cora Pubmed Computers Chameleon Actor Squirrel Texas GPRGNN 17.62ms / 3.74s 20.19ms / 5.53s 39.93ms / 11.40s 16.74ms / 3.40s 19.31ms / 4.49s 25.28ms / 5.12s 17.56ms / 3.55s APPNP 17.16ms / 4.00s 18.47ms / 6.29s 39.59ms / 20.00s 17.01ms / 3.44s 16.32ms / 4.04s 22.93ms / 4.63s 15.96ms / 3.24s MLP 4.14”
How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision (2022, background): “We observe a more complex mixture distribution of homophily and average degree in Flickr and Crocodile, and this seems to be equivalent to mixing graphs of different characteristics, resulting in inconsistent results with our attention design recipe.”
How Powerful are Spectral Graph Neural Networks (2022, methodology): “Settings of these models are detailed in Appendix H. JacobiConv outperforms all existing models on 9 out of 10 datasets and achieves performance gains up to 12% on a heterogeneous dataset Squirrel.”
Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challenges (2022, methodology): “[197] 2023 Federated GNN supporting supervised and unsupervised learning with security measures Cross-device Centralized Decentralized Simulation GCN GAT Custom algorithm Synchronous Squential Facebook [196] LastFM [198]”
Understanding over-squashing and bottlenecks on graphs via curvature (2021, methodology): “We evaluate the methods on nine datasets: Cornell, Texas and Wisconsin from the WebKB dataset 4 ; Chameleon and Squirrel (Rozemberczki et al., 2021) along with Actor (Tang et al., 2009); and Cora (McCallum et al., 2000), Citeseer (Sen et al., 2008) and Pubmed (Namata et al., 2012).”
Geom-GCN: Geometric Graph Convolutional Networks (2020, background): “We classify the nodes into ﬁve categories in term of words of actor’s Wikipedia.”
Is Homophily a Necessity for Graph Neural Networks? (2021, methodology): “Especially, in Texas , there is only 1 node with label 1 .”