Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting

By Defu Cao, Yujing Wang, Juanyong Duan, Ce Zhang, Xia Zhu, Congrui Huang, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong and 1 more. Neural Information Processing Systems, 2020

788 citations recorded by the source corpus.

Multivariate time-series forecasting plays a crucial role in many real-world applications. It is a challenging problem as one needs to consider both intra-series temporal correlations and inter-series correlations simultaneously. Recently, there have been multiple works trying to capture both correlations, but most, if not all of them only capture temporal correlations in the time domain and resort to pre-defined priors as inter-series relationships. In this paper, we propose Spectral Temporal Graph Neural Network (StemGNN) to further improve the accuracy of multivariate time-series forecasting. StemGNN captures inter-series correlations and temporal dependencies jointly in the spectral domain. It combines Graph Fourier Transform (GFT) which models inter-series correlations and Discrete Fourier Transform (DFT) which models temporal dependencies in an end-to-end framework. After passing through GFT and DFT, the spectral representations hold clear patterns and can be predicted effectively by convolution and sequential learning modules. Moreover, StemGNN learns inter-series correlations automatically from the data without using pre-defined priors. We conduct extensive experiments on ten real-world datasets to demonstrate the effectiveness of StemGNN.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/645054d31fa26b29bbfb0cf73b75f8906c359415

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

A Time Series is Worth 64 Words: Long-term Forecasting with Transformers (2022, methodology): “…Possibility of learning spatial correlations across series: Although we haven’t focused on this research in our paper, the channel-independence design can be naturally extended to learn cross-channel relationships by using methods like graph neural networks (Cao et al., 2020; Chen et al., 2021).”
ETA Prediction with Graph Neural Networks in Google Maps (2021, methodology): “This generic blueprint of a spatial GNN component combined with a temporal component yielded several subsequent proposals, including STGCN [34], GaAN [37], Graph WaveNet [29], ST-GRAT [19], StemGNN [3], and GMAN [39].”
Crossformer: Transformer Utilizing Cross-Dimension Dependency for Multivariate Time Series Forecasting (2023, methodology): “…neural models explicitly capture the cross-dimension dependency, i.e. preserving the information of dimensions in the latent feature space and using convolution neural network (CNN) (Lai et al., 2018) or graph neural network (GNN) (Wu et al., 2020; Cao et al., 2020) to capture their dependency.”
Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structures (2022, methodology): “MTGNN [24], StemGNN [4], and IGMTF [25] leverage the graph neural networks to explicitly model the correlations among different time series.”
A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detection (2023, background): “An alternative strand of methodologies,includingStemGNN[50]andTGC[28],focuses onmodelingtemporalcluesinthefrequencydomain.”
Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey (2023, methodology): “For example, StemGNN [176] decomposed the temporal components by the subtraction residual from N-BEATS, but modeled spatial correlations in the spectral domain.”
Frequency-domain MLPs are More Effective Learners in Time Series Forecasting (2023, methodology): “We preprocess all datasets following [16, 13, 14] and normalize them with the min-max normalization.”
TS2Vec: Towards Universal Representation of Time Series (2021, methodology): “For forecasting tasks, the results of TCN, StemGNN and N-BEATS for all datasets and all baselines for Electricity dataset are based on our reproduction.”
Spatial-Temporal Identity: A Simple yet Effective Baseline for Multivariate Time Series Forecasting (2022, methodology): “HI MAE 42.33 42.35 42.37 42.36 49.02 49.03 49.06 49.04 34.55 34.57 34.59 34.57 3.06 3.06 3.05 3.06 92.44 92.58 92.79 92.58 RMSE 61.64 61.66 61.67 61.66 71.16 71.18 71.20 71.18 50.41 50.43 50.44 50.43 7.05 7.05 7.03 7.04 167.00 167.05 167.21 167.07 MAPE 29.90% 29.92% 29.96% 29.92% 22.73% 22.75% 22.79”
Spatio-Temporal Adaptive Embedding Makes Vanilla Transformer SOTA for Traffic Forecasting (2023, background): “…aims to infer the traffic data in the future 𝑇 ′ frames by training a model F (·) with parameters 𝜃 , which can be formulated as: where each frame 𝑋 𝑖 ∈ R 𝑁 × 𝑑 , 𝑁 is the number of spatial nodes. is the dimension of the input feature which equals 1 in our case, standing for traffic volume.”
FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective (2023, neutral): “Since the prior graph structure is usually unknown in most multivariate time series scenarios [10, 2, 13], and the elements of X t are spatially or temporally correlated with each other because of time lag effect [28], we assume all nodes in the hypervariate graph G t are fully-connected.”
Pre-training Enhanced Spatial-temporal Graph Neural Network for Multivariate Time Series Forecasting (2022, neutral): “GraphWaveNet [36], MTGNN [35], STGCN [38], and StemGNN [4] combine graph convolutional networks and gated temporal convolutional networks with their variants.”
Spatio-Temporal Meta-Graph Learning for Traffic Forecasting (2022, neutral): “StemGNN (Cao et al. 2020) took the self-attention (Vaswani et al. 2017) learned from the input as the latent graph.”
Learning Graph Structures With Transformer for Multivariate Time-Series Anomaly Detection in IoT (2021, methodology): “Recent works [17], [26], [4] then combined temporal modeling methods with GCNs to model the topological relationships between sensors.”
Graph Neural Network for Traffic Forecasting: A Survey (2021, neutral): “Problem Relevant Studies Road Traffic Flow [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48], [49], [50], [51], [52], [53], [54], [55], [56], [57], [58], [59], [60], [61], [62],”