A Multi-Horizon Quantile Recurrent Forecaster

By Ruofeng Wen, Kari Torkkola, Balakrishnan Narayanaswamy, Dhruv Madeka. 2017

527 citations recorded by the source corpus.

We propose a framework for general probabilistic multi-step time series regression. Specifically, we exploit the expressiveness and temporal nature of Recurrent Neural Networks, the nonparametric nature of Quantile Regression and the efficiency of Direct Multi-Horizon Forecasting. A new training scheme for recurrent nets is designed to boost stability and performance. We show that the approach accommodates both temporal and static covariates, learning across multiple related series, shifting seasonality, future planned event spikes and cold-starts in real life large-scale forecasting. The performance of the framework is demonstrated in an application to predict the future demand of items sold on Amazon.com, and in a public probabilistic forecasting competition to predict electricity price and load.

Subfields: Artificial Intelligence, Management Science and Operations Research, Signal Processing, Time-Series & Forecasting

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Signal Processing

In Time-Series & Forecasting

Citations in the corpus

Time-series forecasting with deep learning: a survey (2020, methodology): “he loss functions above are the most common across applications, we note that the ﬂexibility of neural networks also allows for more complex losses to be adopted - e.g. losses for quantile regression [56] and multinomial classiﬁcation [32]. Probabilistic Outputs While point estimates are crucial to ”
Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting (2019, methodology): “lti-horizon forecasting [29, 35], recent deep learning methods can be categorized into iterated approaches using autoregressive models [15, 25, 31] or direct methods using sequence-to-sequence models [13, 39]. Iterated approaches utilize one-step-ahead prediction models, with multi-step predictions ”
High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processes (2019, methodology): “We evaluated our approach against DeepAR [13] and MQCNN [22], which we believe are a fair representation of the state-of-the-art in deep-learning-based forecasting.”
Deep Generative Quantile-Copula Models for Probabilistic Forecasting (2019, background): “…be formulated as learning p(yt+d, · · · , yt+1|y:t,x) where y:t denotes the observed series before time t. Multi-horizon quantile forecaster (MQ-RNN/CNN, Wen et al, 2017) combines multi-horizon forecasts, quantile regression and sequence-to-sequence architecture, and predicts multiple quantiles…”
Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting (2019, methodology): “Instead, [19] uses an RNN as an encoder and Multi-layer Perceptrons (MLPs) as a decoder to solve the so-called error accumulation issue and conduct multiahead forecasting in parallel.”
Deep Factors for Forecasting (2019, methodology): “, 2018), MQ-RNN (MR), a sequence model that generates conditional predictive quantiles (Wen et al., 2017), and Prophet (P), a Bayesian structural time series model (Taylor & Letham, 2017).”
Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting (2019, neutral): “Let [ n ] { 1 , 2 , ..., n for a positive integer n . sub-matrix } For I ⊆ [ m ] and J ⊆ [ n ] , the notation M [ , J ] will denote the sub-matrix of M with rows in I and columns in J . means s I M [: , J ] that all the rows are selected and similarly M [ ,”
Recurrent Neural Filters: Learning Independent Bayesian Filtering Steps for Time Series Prediction (2019, methodology): “RNNs for Multistep Prediction: Customised sequenceto-sequence architectures have been explored in [38], [39] for multistep time series prediction, typically predefining”
TSMixer: An all-MLP Architecture for Time Series Forecasting (2023, methodology): “RNN-based models employ gating mechanisms to perform time-domain operations, while the hidden layers in the recurrent cells are used to perform the feature-domain operations (Wen et al., 2017; Salinas et al., 2020).”
Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting (2020, methodology): “However, existing methods are designed under limited problem setting, like predicting 48 points or less (Hochreiter and Schmidhuber 1997; Li et al. 2018; Yu et al. 2017; Liu et al. 2019; Qin et al. 2017; Wen et al. 2017).”
TS2Vec: Towards Universal Representation of Time Series (2021, methodology): “Time Series Forecasting Deep learning methods including RNNs (Salinas et al. 2020; Wen et al. 2017; Oreshkin et al. 2019), CNNs (Bai, Kolter, and Koltun 2018; Wan et al. 2019), GNNs (Cao et al. 2020) and Transformers (Li et al. 2019; Zhou et al. 2021), have been widely applied to time series…”
Probabilistic Forecasting with Temporal Convolutional Neural Network (2019, background): “Both RNNs and CNNs have been shown to be able to model complex nonlinear feature interactions and yield substantial forecasting performances, especially when many related time series are available (Smyl, 2016; Laptev et al., 2017; Wen et al., 2017; Salinas et al., 2019; Rangapuram et al., 2018).”
Time Series Data Augmentation for Deep Learning: A Survey (2020, methodology): “In this subsection we demonstrate the practical effectiveness of data augmentation techniques in three models, MQRNN [Wen et al., 2017], DeepAR [Salinas et al.”
A decoder-only foundation model for time-series forecasting (2023, background): “(6) Note that if one is interested in probabilistic forecasting, then it is easy to have multiple output heads for each output patch, each head minimizing a separate quantile loss as in (Wen et al., 2017).”
Forecasting: theory and practice (2020, methodology): “Recently, to deal with these new challenges, numerous empirical studies have identified the potentials of deep learning based global models, in both point and probabilistic demand forecasting (e.g., Wen et al., 2017; Rangapuram et al., 2018; Salinas et al., 2019a; Bandara et al., 2020b).”
Deep State Space Models for Time Series Forecasting (2018, background): “With their capability to extract higher order features, they can identify complex patterns within and across time series, and can do so from datasets of raw time series with considerably less human effort [9, 27, 19].”
Deep Factors with Gaussian Processes for Forecasting (2018, background): “Deep neural networks (DNNs), in particular, recurrent neural networks (RNNs), such as LSTMs [6] have been successful in time series forecasting [7, 8].”
A Unified Framework for Long Range and Cold Start Forecasting of Seasonal Profiles in Time Series (2017, background): “The long-range forecasting problem has traditionally been understudied, though see [1, 2] for recent approaches.”
N-HiTS: Neural Hierarchical Interpolation for Time Series Forecasting (2022, background): “Aside from its convenient bias-variance benefits and robustness (Mar-cellino, Stock, and Watson 2006; Atiya and Taieb 2016), the multi-step strategy has enabled the models to efficiently predict long sequences in a single forward pass (Wen et al. 2017; Zhou et al. 2020; Lim et al. 2021).”
CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting (2022, background): “Recently, there has been a surge of efforts applying deep learning for forecasting (Wen et al., 2017; Bai et al., 2018; Zhou et al., 2021), and owing to the increase in data availability and computational resources, these approaches have offered promising performance over conventional methods in…”