Unsupervised Pre-training of a Deep LSTM-based Stacked Autoencoder for Multivariate Time Series Forecasting Problems
By Alaa El. Sagheer, Mostafa Kotb. Scientific Reports, 2019
302 citations recorded by the source corpus.
Currently, most real-world time series datasets are multivariate and are rich in dynamical information of the underlying system. Such datasets are attracting much attention; therefore, the need for accurate modelling of such high-dimensional datasets is increasing. Recently, the deep architecture of the recurrent neural network (RNN) and its variant long short-term memory (LSTM) have been proven to be more accurate than traditional statistical methods in modelling time series data. Despite the reported advantages of the deep LSTM model, its performance in modelling multivariate time series (MTS) data has not been satisfactory, particularly when attempting to process highly non-linear and long-interval MTS datasets. The reason is that the supervised learning approach initializes the neurons randomly in such recurrent networks, disabling the neurons that ultimately must properly learn the latent features of the correlated variables included in the MTS dataset. In this paper, we propose a pre-trained LSTM-based stacked autoencoder (LSTM-SAE) approach in an unsupervised learning fashion to replace the random weight initialization strategy adopted in deep LSTM recurrent networks. For evaluation purposes, two different case studies that include real-world datasets are investigated, where the performance of the proposed approach compares favourably with the deep LSTM approach. In addition, the proposed approach outperforms several reference models investigating the same case studies. Overall, the experimental results clearly show that the unsupervised pre-training approach improves the performance of deep LSTM and leads to better and faster convergence than other models.
Subfields: Electrical and Electronic Engineering, Management Science and Operations Research, Signal Processing
View Paper: https://www.semanticscholar.org/paper/9e39783012e535f36f12ce0008e1986c0e514261
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Electrical and Electronic Engineering
In Management Science and Operations Research
In Signal Processing
Citations in the corpus
- PPFL: privacy-preserving federated learning with trusted execution environments (2021, methodology): “For example, one can perform layer-wise training on (i) Graph Neural Networks by disentangling feature aggregation and feature transformation [74], and (ii) Long Short-Term Memory networks (LSTMs), by adding hidden layers [60].”