A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction

By Yao Qin, Dongjin Song, Haifeng Chen, Wei Cheng, Guofei Jiang, Garrison W. Cottrell. 2017

1,569 citations recorded by the source corpus.

The Nonlinear autoregressive exogenous (NARX) model, which predicts the current value of a time series based upon its previous values as well as the current and past values of multiple driving (exogenous) series, has been studied for decades. Despite the fact that various NARX models have been developed, few of them can capture the long-term temporal dependencies appropriately and select the relevant driving series to make predictions. In this paper, we propose a dual-stage attention-based recurrent neural network (DA-RNN) to address these two issues. In the first stage, we introduce an input attention mechanism to adaptively extract relevant driving series (a.k.a., input features) at each time step by referring to the previous encoder hidden state. In the second stage, we use a temporal attention mechanism to select relevant encoder hidden states across all time steps. With this dual-stage attention scheme, our model can not only make predictions effectively, but can also be easily interpreted. Thorough empirical studies based upon the SML 2010 dataset and the NASDAQ 100 Stock dataset demonstrate that the DA-RNN can outperform state-of-the-art methods for time series prediction.

Subfields: Artificial Intelligence, Management Science and Operations Research, Model Architectures, Signal Processing, Time-Series & Forecasting

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mingsheng Long
  2. Jianmin Wang
  3. Jindong Wang
  4. Tao Qin
  5. Dacheng Tao
  6. Jiashi Feng
  7. Kate Saenko
  8. Junchi Yan

In Management Science and Operations Research

  1. Haixu Wu
  2. Mingsheng Long
  3. Jianmin Wang
  4. Junchi Yan
  5. Bernie Wang
  6. Tim Januschowski
  7. Jan Gasthaus
  8. Liang Sun

In Model Architectures

  1. Tao Qin
  2. Weiwei Jiang
  3. Bernie Wang
  4. Slawek Smyl
  5. Tao Lin
  6. Yao Zhao
  7. Cesare Alippi
  8. Junchi Yan

In Signal Processing

  1. Mingsheng Long
  2. Haixu Wu
  3. Jianmin Wang
  4. Liang Sun
  5. Junchi Yan
  6. Bernie Wang
  7. Qingsong Wen
  8. Tian Zhou

In Time-Series & Forecasting

  1. Bernie Wang
  2. Jan Gasthaus
  3. Tim Januschowski
  4. Haixu Wu
  5. Mingsheng Long
  6. Jianmin Wang
  7. Syama Sundar Rangapuram
  8. David Salinas

Citations in the corpus