Dynamic Network Embedding : An Extended Approach for Skip-gram based Network Embedding

By Lun Du, Yun Wang, Guojie Song, Zhicong Lu, Junshan Wang. 2018

231 citations recorded by the source corpus.

Network embedding, as an approach to learn low-dimensional representations of vertices, has been proved extremely useful in many applications. Lots of state-of-the-art network embedding methods based on Skip-gram framework are efficient and effective. However, these methods mainly focus on the static network embedding and cannot naturally generalize to the dynamic environment. In this paper, we propose a stable dynamic embedding framework with high efficiency. It is an extension for the Skip-gram based network embedding methods, which can keep the optimality of the objective in the Skip-gram based methods in theory. Our model can not only generalize to the new vertex representation, but also update the most affected original vertex representations during the evolvement of the network. Multi-class classification on three real-world networks demonstrates that, our model can update the vertex representations efficiently and achieve the performance of retraining simultaneously. Besides, the visualization experimental result illustrates that, our model is capable of avoiding the embedding space drifting.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

Citations in the corpus

Unsupervised Inductive Whole-Graph Embedding by Preserving Graph Proximity (2019, methodology): “DEEPWALK is also known as “skip-gram based methods” (Du et al., 2018) due to its use of WORD2VEC.”
Recurrent Event Network for Reasoning over Temporal Knowledge Graphs (2019, background): “Furthermore, extending existing dynamic network embedding models [8, 29, 16, 8] to perform TKG reasoning is non-trivial—they do not capture multi-relational entity interaction patterns and require expensive re-training when doing multi-step prediction over future time stamps.”
Deep Representation Learning for Social Network Analysis (2019, background): “Rows of the proximity matrix S ∈ R|V|×|V| are fed into the autoencoder to learn and generate embeddings Z ∈ R|V|×D at the hidden layer. of SDNE is: L = ∑ vi∈V ||(ψdec(zi)−Si∗)⊙bi|| 2 2+ |V|∑ i,j=1 Sij||zi−zj|| 2 2, 9tar = {Si∗, Sij}.”
Temporal Graph Networks for Deep Learning on Dynamic Graphs (2020, background): “Another line of work encodes DTDGs by ﬁrst performing random walks on an initial snapshot and then modifying the walk behaviour for subsequent snapshots [40, 14, 64, 13, 71].”
AliGraph: A Comprehensive Graph Neural Network Platform (2019, methodology): “Considering the new vertices’ influence on the original networks, [11] extends the skip-gram methods to update the original vertices’ embedding.”
Streaming Graph Neural Networks via Continual Learning (2020, methodology): “For LINE and DNE, we use both first-order and second-order proximity representations and concatenate them.”
Relational Representation Learning for Dynamic (Knowledge) Graphs: A Survey (2019, unknown): “Another notable work where time is used as a regularizer is an extension of a well-known model for static graphs, named LINE [214], to DTDGs by Du et al. [65].”
Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks (2021, background): “Network-embedding approaches may also be applied to temporal networks (Zhou et al., 2018; Du et al., 2018; Mahdavi et al., 2018; Singer et al., 2019; Nguyen et al., 2018).”
Representation Learning for Dynamic Graphs: A Survey (2019, methodology): “(2015)), to DTDGs by Du et al. (2018). Besides using time as regularizer, the authors propose a way of recomputing the node embeddings only for the nodes that have been influenced greatly from the last snapshot.”
DySAT: Deep Neural Representation Learning on Dynamic Graphs via Self-Attention Networks (2020, background): “A related problem is representation learning in streaming graphs where the objective is to improve efficiency over repeatedly retraining static models, by proposing incremental temporal updates [4, 17, 18, 38].”