Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting

By Rajat Sen, Hsiang-Fu Yu, Inderjit S. Dhillon. 2019

473 citations recorded by the source corpus.

Forecasting high-dimensional time series plays a crucial role in many applications such as demand forecasting and financial predictions. Modern datasets can have millions of correlated time-series that evolve together, i.e they are extremely high dimensional (one dimension for each individual time-series). There is a need for exploiting global patterns and coupling them with local calibration for better prediction. However, most recent deep learning approaches in the literature are one-dimensional, i.e, even though they are trained on the whole dataset, during prediction, the future forecast for a single dimension mainly depends on past values from the same dimension. In this paper, we seek to correct this deficiency and propose DeepGLO, a deep forecasting model which thinks globally and acts locally. In particular, DeepGLO is a hybrid model that combines a global matrix factorization model regularized by a temporal convolution network, along with another temporal network that can capture local properties of each time-series and associated covariates. Our model can be trained effectively on high-dimensional but diverse time series, where different time series can have vastly different scales, without a priori normalization or rescaling. Empirical results demonstrate that DeepGLO can outperform state-of-the-art approaches; for example, we see more than 25% improvement in WAPE over other methods on a public dataset that contains more than 100K-dimensional time series.

Subfields: Management Science and Operations Research, Signal Processing, Time-Series & Forecasting

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Management Science and Operations Research

In Signal Processing

In Time-Series & Forecasting

Citations in the corpus

High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processes (2019, methodology): “Similar sequence modeling techniques have also been used in the context of forecasting to make probabilistic predictions for collections of real or integer-valued time series [26, 36, 16].”
Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions (2019, methodology): “The implemented models are standard RNN architectures with adequate preprocessing without considering special and sophisticated network types such as in Smyl (2020)’s work involving RNNs.”
FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective (2023, unknown): “…a comprehensive comparison of the forecasting performance between our FourierGNN and several representative and state-of-the-art (SOTA) models on the seven datasets, including classic method VAR [32], deep learning-based models such as SFM [24], LSTNet [33], TCN [8], DeepGLO [34], and CoST [36].”
A decoder-only foundation model for time-series forecasting (2023, methodology): “Such models include the classical VAR model (Zivot & Wang, 2006) as well as deep learning models like (Sen et al., 2019; Zhou et al., 2022; 2021) to name a few.”
MICN: Multi-scale Local and Global Context Modeling for Long-term Series Forecasting (2023, background): “It can integrate the local information of the sequence better and achieve competitive results in short and medium-term forecasting (Sen et al., 2019) (Borovykh et al.”
Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting (2020, background): “Some efforts employ temporal convolutional networks [21, 22, 23] to enable the model process very long sequence with fewer time.”
Frequency-domain MLPs are More Effective Learners in Time Series Forecasting (2023, methodology): “For short-term forecasting, we compre FreTS against VAR [23], SFM [29], LSTNet [10], TCN [11], GraphWaveNet [27], DeepGLO [37], StemGNN [16], MTGNN [15], and AGCRN [17] for comparison.”
Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting (2020, methodology): “We compare the performances of StemGNN on nine public datasets, ranging from traffic, energy and electrocardiogram domains with other state-of-the-art models, including FC-LSTM [26], SFM [32], N-BEATS [19], DCRNN [17], LSTNet [14], ST-GCN [31], DeepState [21], TCN [3], Graph Wavenet [29] and DeepGLO”
Deep Learning on Traffic Prediction: Methods, Analysis, and Future Directions (2021, methodology): “[ 84] developed a hybrid model that associated a global matrix decomposition model regularized by a temporal deep network with a local deep temporal model that captured patterns speciﬁc to each dimension.”
ModernTCN: A Modern Pure Convolution Structure for General Time Series Analysis (2024, background): “For example, TCN and its variants (Bai et al., 2018; Sen et al., 2019; Franceschi et al., 2019) adopt causal convolution to model the DWConv and PWConv are short for depth-wise and point-wise convolution (2017).”
AdaRNN: Adaptive Learning and Forecasting of Time Series (2021, methodology): “These deep methods tackle TS classification or forecasting by leveraging attention mechanisms [23, 33] or tensor factorization [40] for capturing shared information among series.”
Deep Learning for Time Series Forecasting: Tutorial and Literature Survey (2020, methodology): “DeepGLO [162] CNN Point ≥ 1 l2 WAPE, MAPE, sMAPE electricity, traffic, wiki Global matrix factorization regularized by a deep leveled network”
FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting (2022, background): “The temporal convolutional network (Sen et al., 2019) is another family efﬁcient in sequential tasks.”
FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecasting (2022, methodology): “where the definition of A and B can be found in Voelker et al. (2019). Using Legendre Polynomials as a basis allows us to accurately approximate smooth functions, as indicated by the following theorem.”
SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interaction (2021, methodology): “Over the years, TCN has been widely used in all kinds of time series forecasting problems and achieve promising results [39, 33].”
From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting (2020, methodology): “Several classic techniques such as Autoregressive Moving Average Models (ARMA) [43] have been incorporated in deep learning architectures [41, 18] in modern day state-of-the-art time series forecasting methods [37].”
Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts (2024, unknown): “…2023; Zhou et al., 2021; Nie et al., 2023; Liu et al., 2024b; Wang et al., 2024c; Chen et al., 2024; Wang et al., 2022) and non-transformer models (Sen et al., 2019; Jin et al., 2022; Wang et al., 2024b; Hu et al., 2024; Qi et al., 2024; Wang, 2024), designed to handle multiple time series…”