Temporal Graph Networks for Deep Learning on Dynamic Graphs

By Emanuele Rossi, Benjamin Paul Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti, Michael M. Bronstein. arXiv.org, 2020

1,099 citations recorded by the source corpus.

Graph Neural Networks (GNNs) have recently become increasingly popular due to their ability to learn complex systems of relations or interactions arising in a broad spectrum of problems ranging from biology and particle physics to social networks and recommendation systems. Despite the plethora of different models for deep learning on graphs, few approaches have been proposed thus far for dealing with graphs that present some sort of dynamic nature (e.g. evolving features or connectivity over time). In this paper, we present Temporal Graph Networks (TGNs), a generic, efficient framework for deep learning on dynamic graphs represented as sequences of timed events. Thanks to a novel combination of memory modules and graph-based operators, TGNs are able to significantly outperform previous approaches being at the same time more computationally efficient. We furthermore show that several previous models for learning on dynamic graphs can be cast as specific instances of our framework. We perform a detailed ablation study of different components of our framework and devise the best configuration that achieves state-of-the-art performance on several transductive and inductive prediction tasks for dynamic graphs.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/21ab210e28cb37e9ef0241850649a2963791f124

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

Citations in the corpus

Attention, please! A survey of neural attention models in deep learning (2021, background): “To improve the representation of nodes and expand the capacity of GATs to deal with data of a dynamic nature (i.e. evolving features or connectivity 1 https:// github. com/ lucid rains/ DALLE- pytor ch. 1 3 over time), architectures that combine memory modules and the temporal dimension, like MGNs a”
Graph Representation Learning in Biomedicine and Healthcare (2021, positive): “Recent architectures have leveraged dynamic message passing 72,85,86 to deal with evolving and time-varying graphs, as well as few-shot learning 87 or self-supervised strategies 88,89 to deal with graphs that are poorly annotated and that have limited information about labels.”
Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks (2021, background): “Temporal Graph Networks(TGN) (Rossi et al., 2020) is a recent work that performs representation learning on temporal networks, which has been developed in parallel with this work.”
ETA Prediction with Graph Neural Networks in Google Maps (2021, background): “A recent line of research on dynamic graph representation learning [20, 30] tries to explicitly account for such situations, both on the featurisation and the topological side.”
Foundations and Modeling of Dynamic Networks Using Dynamic Graph Neural Networks: A Survey (2020, methodology): “Link prediction may for example be applied in knowledge graph completion [21], [22] or by recommender systems [18], [19].”