Inductive Representation Learning on Temporal Graphs

By Da Xu, Chuanwei Ruan, Evren Körpeoglu, Sushant Kumar, Kannan Achan. International Conference on Learning Representations, 2020

989 citations recorded by the source corpus.

Inductive representation learning on temporal graphs is an important step toward salable machine learning on real-world dynamic networks. The evolving nature of temporal dynamic graphs requires handling new nodes as well as capturing temporal patterns. The node embeddings, which are now functions of time, should represent both the static node features and the evolving topological structures. Moreover, node and topological features can be temporal as well, whose patterns the node embeddings should also capture. We propose the temporal graph attention (TGAT) layer to efficiently aggregate temporal-topological neighborhood features as well as to learn the time-feature interactions. For TGAT, we use the self-attention mechanism as building block and develop a novel functional time encoding technique based on the classical Bochner's theorem from harmonic analysis. By stacking TGAT layers, the network recognizes the node embeddings as functions of time and is able to inductively infer embeddings for both new and observed nodes as the graph evolves. The proposed approach handles both node classification and link prediction task, and can be naturally extended to include the temporal edge features. We evaluate our method with transductive and inductive tasks under temporal settings with two benchmark and one industrial dataset. Our TGAT model compares favorably to state-of-the-art baselines as well as the previous temporal graph embedding approaches.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Transportation

View Paper: https://www.semanticscholar.org/paper/1f58e8d4c827037d4c2a1afc695a88704e088beb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

In Transportation

Citations in the corpus

ETA Prediction with Graph Neural Networks in Google Maps (2021, background): “A recent line of research on dynamic graph representation learning [20, 30] tries to explicitly account for such situations, both on the featurisation and the topological side.”
Temporal Graph Networks for Deep Learning on Dynamic Graphs (2020, methodology): “The gap is particularly large on the Twitter dataset, where we outperfom the second-best method (TGAT) by over 25%.”
Foundations and Modeling of Dynamic Networks Using Dynamic Graph Neural Networks: A Survey (2020, methodology): “Link prediction may for example be applied in knowledge graph completion [21], [22] or by recommender systems [18], [19].”
Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks (2021, methodology): “Then, it is easy to show that ICAW (a; {Sa, Sb}) 6= ICAW (a′; {Sa′ , Sb}) that allows differentiating a and a′, while TGAT (Xu et al., 2020) as discussed in Sec.”
A Survey on Knowledge Graphs: Representation, Acquisition, and Applications (2020, background): “For example, the temporal graph attention (TGAT) network [174] that captures temporal-topological structure and”