EA-LSTM: Evolutionary Attention-based LSTM for Time Series Prediction

By Youru Li, Zhenfeng Zhu, Deqiang Kong, Hua Han, Yao Zhao. Knowledge-Based Systems, 2018

448 citations recorded by the source corpus.

Time series prediction with deep learning methods, especially long short-term memory neural networks (LSTMs), have scored significant achievements in recent years. Despite the fact that the LSTMs can help to capture long-term dependencies, its ability to pay different degree of attention on sub-window feature within multiple time-steps is insufficient. To address this issue, an evolutionary attention-based LSTM training with competitive random search is proposed for multivariate time series prediction. By transferring shared parameters, an evolutionary attention learning approach is introduced to the LSTMs model. Thus, like that for biological evolution, the pattern for importance-based attention sampling can be confirmed during temporal relationship mining. To refrain from being trapped into partial optimization like traditional gradient-based methods, an evolutionary computation inspired competitive random search method is proposed, which can well configure the parameters in the attention layer. Experimental results have illustrated that the proposed model can achieve competetive prediction performance compared with other baseline methods.

Subfields: Artificial Intelligence, Management Science and Operations Research, Signal Processing

View Paper: https://www.semanticscholar.org/paper/f601f077a43056ee66d52e2a78063c81b321e3bc

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Huang Yipan
P. T. Yamak
Ke Zhang
Baiping Chen
Yifei Zhu
Jin Fan
Li Yujian
P. K. Gadosey

Citations in the corpus

Parallel spatio-temporal attention-based TCN for multivariate time series prediction (2021, background): “Presently, recurrent neural networks (RNNs) are the go-to approach for multivariate time series prediction [15], [16].”
A Comparison between ARIMA, LSTM, and GRU for Time Series Forecasting (2019, background): “[11] research aim to address the problem of LSTMs insufficient capacity to pay different degree of attention on a sub-window feature within multiple time-steps.”