Position-Based Content Attention for Time Series Forecasting with Sequence-to-Sequence RNNs

By Yagmur Gizem Cinar, Hamid Mirisaee, Parantapa Goswami, Éric Gaussier, Ali Aït-Bachir, Vadim V. Strijov. 2017

83 citations recorded by the source corpus.

We propose here an extended attention model for sequence-to-sequence recurrent neural networks (RNNs) designed to capture (pseudo-)periods in time series. This extended attention model can be deployed on top of any RNN and is shown to yield state-of-the-art performance for time series forecasting on several univariate and multivariate time series.

Subfields: Artificial Intelligence, Management Science and Operations Research, Model Architectures, Signal Processing, Time-Series & Forecasting

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jan Gasthaus
2. Bernie Wang
3. Tim Januschowski
4. Laurent Callot
5. David Salinas
6. Syama Sundar Rangapuram
7. Danielle C. Maddix
8. Chenyou Fan

In Management Science and Operations Research

1. Bernie Wang
2. Tim Januschowski
3. Jan Gasthaus
4. Syama Sundar Rangapuram
5. Laurent Callot
6. Dhruv Madeka
7. Danielle C. Maddix
8. David Salinas

In Model Architectures

1. Bernie Wang
2. Dhruv Madeka
3. Laurent Callot
4. Michael Bohlke-Schneider
5. David Salinas
6. Jian Pei
7. Heng Huang
8. Kari Torkkola

In Signal Processing

1. Bernie Wang
2. Jan Gasthaus
3. Tim Januschowski
4. Syama Sundar Rangapuram
5. Laurent Callot
6. Danielle C. Maddix
7. Dhruv Madeka
8. David Salinas

In Time-Series & Forecasting

1. Bernie Wang
2. Jan Gasthaus
3. Tim Januschowski
4. Syama Sundar Rangapuram
5. Danielle C. Maddix
6. Dhruv Madeka
7. Bryan Lim
8. David Salinas

Citations in the corpus

• A Multi-Horizon Quantile Recurrent Forecaster (2017, background): “Such architecture is intimately related to multi-step time series forecasting, a connection which has been well investigated in recent studies (Cinar et al, 2017 and Flunkert et al, 2017).”
• Exploring Interpretable LSTM Neural Networks over Multi-Variable Data (2019, unknown): “Attention mechanism has gained tremendous popularity (Xu et al., 2018; Choi et al., 2018; Guo et al., 2018; Lai et al., 2017; Qin et al., 2017; Cinar et al., 2017; Choi et al., 2016; Vinyals et al., 2015; Bahdanau et al., 2014).”
• A survey on long short-term memory networks for time series prediction (2021, methodology): “An approach for the integration of attention to optimize the copying procedure is presented by [34].”
• Deep Learning for Time Series Forecasting: Tutorial and Literature Survey (2020, background): “[39] RNN + Attention Point ≥ 1 l2 , l2 regularizer MSE, sMAPE energy, max temperature, CPU usage, air quality Attention mechanism on top of RNN”
• Multi-Horizon Time Series Forecasting with Temporal Attention Learning (2019, methodology): “POS-RNN produced a significant lower quantile loss of 2.95, by exploring historical patterns with attention mechanism.”