Deep Factors for Forecasting

By Bernie Wang, Alex Smola, Danielle C. Maddix, Jan Gasthaus, Dean Phillips Foster, Tim Januschowski. 2019

205 citations recorded by the source corpus.

Producing probabilistic forecasts for large collections of similar and/or dependent time series is a practically relevant and challenging task. Classical time series models fail to capture complex patterns in the data, and multivariate techniques struggle to scale to large problem sizes. Their reliance on strong structural assumptions makes them data-efficient, and allows them to provide uncertainty estimates. The converse is true for models based on deep neural networks, which can learn complex patterns and dependencies given enough data. In this paper, we propose a hybrid model that incorporates the benefits of both approaches. Our new method is data-driven and scalable via a latent, global, deep component. It also handles uncertainty through a local classical model. We provide both theoretical and empirical evidence for the soundness of our approach through a necessary and sufficient decomposition of exchangeable time series into a global and a local part. Our experiments demonstrate the advantages of our model both in term of data efficiency, accuracy and computational complexity.

Subfields: Artificial Intelligence, Management Science and Operations Research, Signal Processing, Time-Series & Forecasting

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Signal Processing

In Time-Series & Forecasting

Citations in the corpus

Time-series forecasting with deep learning: a survey (2020, methodology): “In general, hybrid models use deep neural networks in two manners: a) to encode time-varying parameters for non-probabilistic parametric models [64,65,67], and b) to produce distribution parameters for probabilistic models [38,40,66].”
N-BEATS: Neural basis expansion analysis for interpretable time series forecasting (2019, neutral): “2 Split used in Deep Factors (Wang et al., 2019).”
Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting (2019, methodology): “Deep State-Space Models (DSSM) [31] adopt a similar approach, utilizing LSTMs to generate parameters of a predefined linear state-space model with predictive distributions produced via Kalman filtering – with extensions for multivariate time series data in [38].”
Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting (2019, methodology): “There has been a concurrent work [25], where an RNN has been used to evolve a global state common to all time-series.”
GluonTS: Probabilistic Time Series Models in Python (2019, background): “, 2003) and Deep Factor models (Maddix et al., 2018; Wang et al., 2019) which we omit due to space restrictions.”
A Survey on Bayesian Deep Learning (2016, methodology): “BDL-based forecasting models [21, 27, 90, 124] achieve these with an RNN-based perception component and a task-specific component handling the conditional dependencies among different variables, showing substantial improvement over previous non-BDL forecasting models.”
Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey (2023, background): “In addition to the spatial perspective, some deep time series models have incorporated statistical modeling techniques to enhance the understanding of predictive outcomes[218].”
Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions (2019, background): “For the number of hidden layers in the RNN, many recent studies suggest that a low value usually performs better (Smyl and Kuber, 2016; Salinas et al., 2019; Wang et al., 2019; Bandara et al., 2020).”
Bayesian Temporal Factorization for Multidimensional Time Series Prediction (2019, background): “Third, we would like to integrate recent advances in deep learning to better capture the complex and non-linear dynamics in modern time series data [39], [40], [41], [42], [43], [44], [45], [46].”
Deep Learning for Time Series Forecasting: Tutorial and Literature Survey (2020, background): “Bandara et al. [13] RNN + Classical Decomposition Point ≥ 1 - sMAPE CIF2016, NN5 Clusters time series based on set of features and train one model per cluster LSTM-MSNet [15] RNN + Classical Decomposition Point ≥ 1 `1 sMAPE, MASE M4, energy Decomposition based model with multiple seasonal patterns C”
Deep Learning for Forecasting (2020, methodology): “Furthermore, we will dive into recent work that combines neural networks with probabilistic models such as deep state space [4] and deep factor [9] models, and bayesian techniques such as [2].”