Pre-training Enhanced Spatial-temporal Graph Neural Network for Multivariate Time Series Forecasting

By Zezhi Shao, Zhao Zhang, Fei Wang, Yongjun Xu. Knowledge Discovery and Data Mining, 2022

370 citations recorded by the source corpus.

Multivariate Time Series (MTS) forecasting plays a vital role in a wide range of applications. Recently, Spatial-Temporal Graph Neural Networks (STGNNs) have become increasingly popular MTS forecasting methods. STGNNs jointly model the spatial and temporal patterns of MTS through graph neural networks and sequential models, significantly improving the prediction accuracy. But limited by model complexity, most STGNNs only consider short-term historical MTS data, such as data over the past one hour. However, the patterns of time series and the dependencies between them (i.e., the temporal and spatial patterns) need to be analyzed based on long-term historical MTS data. To address this issue, we propose a novel framework, in which STGNN is Enhanced by a scalable time series Pre-training model (STEP). Specifically, we design a pre-training model to efficiently learn temporal patterns from very long-term history time series (e.g., the past two weeks) and generate segment-level representations. These representations provide contextual information for short-term time series input to STGNNs and facilitate modeling dependencies between time series. Experiments on three public real-world datasets demonstrate that our framework is capable of significantly enhancing downstream STGNNs, and our pre-training model aptly captures temporal patterns.

Subfields: Building and Construction, Electrical and Electronic Engineering, Signal Processing

View Paper: https://www.semanticscholar.org/paper/e3e721fd51a5940273759c5058b2046740a54d5c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Building and Construction

1. Junbo Zhang
2. Yu Zheng
3. Yuxuan Liang
4. Zezhi Shao
5. Fei Wang
6. Zhao Zhang
7. Yongjun Xu
8. Cesare Alippi

In Electrical and Electronic Engineering

1. Cesare Alippi
2. Yu Zheng
3. Qingsong Wen
4. Junbo Zhang

In Signal Processing

1. Junbo Zhang
2. Geoffrey I. Webb
3. Cesare Alippi
4. Zezhi Shao
5. Yu Zheng
6. Shirui Pan
7. Qingsong Wen
8. Xing Xie

Citations in the corpus

• Spatio-Temporal Adaptive Embedding Makes Vanilla Transformer SOTA for Traffic Forecasting (2023, background): “Researchers have devoted considerable effort to developing fancy and complicated models for traffic forecasting, such as novel graph convolutions [2, 3, 8–10, 12, 17, 20, 27, 30, 31, 36, 40], learning graph structures [14, 23, 33, 34, 39], efficient attention mechanisms [5, 19, 32, 42, 43], and othe”
• A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detection (2023, methodology): “Notable examples include Panagopoulos et al.’s meta-learning for COVID-19 prediction in data-limited settings [200], and Shao et al.’s pre-training framework for spatial-temporal GNNs [61].”
• Decoupled Dynamic Spatial-Temporal Graph Neural Network for Traffic Forecasting (2022, neutral): “With the availability of large-scale traffic data and the rise of artificial intelligence [42], Spatial-Temporal Graph Neural Networks (STGNNs) are proposed tomodel the complex spatial-temporal correlations in traffic data [21, 26, 29, 45, 49].”
• Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey (2023, methodology): “In the most recent work, STEP [246] proposed a pre-training model combined with the Mask Auto-”
• Spatio-Temporal Meta-Graph Learning for Traffic Forecasting (2022, neutral): “However, due to the reproducibility problem ( ⋆ in Table 2), we report the results of GMAN, GTS, and PM-MemNet either from our own experiments or other literature (e.g., GMAN on METR-LA (Shao et al. 2022)).”
• Spatial-Temporal Identity: A Simple yet Effective Baseline for Multivariate Time Series Forecasting (2022, background): “Recently, Spatial-Temporal Graph Neural Networks (STGNNs) [14, 15, 18] have attracted increasing attention for their state-of-the-art performance.”