LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection

By Pankaj Malhotra, Anusha Ramakrishnan, Gaurangi Anand, Lovekesh Vig, Puneet Agarwal, Gautam M. Shroff. arXiv.org, 2016

1,222 citations recorded by the source corpus.

Mechanical devices such as engines, vehicles, aircrafts, etc., are typically instrumented with numerous sensors to capture the behavior and health of the machine. However, there are often external factors or variables which are not captured by sensors leading to time-series which are inherently unpredictable. For instance, manual controls and/or unmonitored environmental conditions or load may lead to inherently unpredictable time-series. Detecting anomalies in such scenarios becomes challenging using standard approaches based on mathematical models that rely on stationarity, or prediction models that utilize prediction errors to detect anomalies. We propose a Long Short Term Memory Networks based Encoder-Decoder scheme for Anomaly Detection (EncDec-AD) that learns to reconstruct 'normal' time-series behavior, and thereafter uses reconstruction error to detect anomalies. We experiment with three publicly available quasi predictable time-series datasets: power demand, space shuttle, and ECG, and two real-world engine datasets with both predictive and unpredictable behavior. We show that EncDec-AD is robust and can detect anomalies from predictable, unpredictable, periodic, aperiodic, and quasi-periodic time-series. Further, we show that EncDec-AD is able to detect anomalies from short time-series (length as small as 30) as well as long time-series (length as large as 500).

Subfields: Artificial Intelligence, Computer Networks and Communications, Signal Processing

View Paper: https://www.semanticscholar.org/paper/e9672150c4f39ab64876e798a94212a93d1770fe

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Signal Processing

Citations in the corpus

Robust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network (2019, methodology): “To demonstrate the effectiveness of OmniAnomaly, we first compare it with four state-of-the-art unsupervised approaches for multivariate time series anomaly detection: LSTM with nonparametric dynamic thresholding (LSTM-NDT for short) [6], EncDec-AD [11], DAGMM [27], and LSTM-VAE [16], which have bee”
Deep Learning for Unsupervised Insider Threat Detection in Structured Cybersecurity Data Streams (2017, methodology): “…user activ-Figure Recurrent neural networks have, of course, been successfully applied to anomaly detection in various alternative domains; e.g., Malhotra et al. (2016) in the domain of signals from mechanical sensors for machinery such as engines, and vehicles, Chuahan et al. (2015) in the…”
Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding (2018, methodology): “These advantages have motivated the use of LSTM networks in several recent anomaly detection tasks [5, 10, 30, 31, 33, 44], where LSTM models are fit on nominal data and model predictions are compared to actual data stream values using a set of detection rules in order to detect anomalies [5, 30, 31”
Deep Learning for Anomaly Detection (2020, methodology): “The most straightforward way is to follow the same procedure as the conventional use of AEs by adapting the network architecture to the type of input data, such as CNN-AE [56, 172], LSTM-AE [96], Conv-LSTM-AE [92], and graph convolutional network-AE [37].”
Multi-head CNN-RNN for multi-time series anomaly detection: An industrial case study (2019, methodology): “Recent approaches use DL algorithms such as Long-Short Term Memory (LSTM) to model the normal behavior of a sequence to create a predictive model and then calculate the anomaly score of an observation as the deviation from the predicted value [43, 44, 45, 46, 47, 48].”
Anomaly Detection in Automobile Control Network Data with Long Short-Term Memory Networks (2016, result): “When the series is inherently unpredictable, the LSTM encoder-decoder can produce superior results to the LSTM predictor [12].”
Anomaly Detection in Time Series: A Comprehensive Evaluation (2022, methodology): “NormalizingFlow [116] DeepAnT [94] STORN[123] CoalESN[99] DeepNAP [72] DeepLSTM[31] HybridKNN [124] EncDec-AD [88] LSTM-basedVAE-GAN[98] LAMP[166] DAE [117] MultiHTM[146] TCN-AE[135] AD-LTI [148] LSTM-AD [89] AE [117] VELC[158] Bagel [79] HealthESN [32] MTAD-GAT [161] MoteESN[30] RADM[40] MSCRED…”
Learning Graph Structures With Transformer for Multivariate Time-Series Anomaly Detection in IoT (2021, methodology): “The LSTMbased encoder-decoder [32] reconstructed the variable values and measured the reconstruction errors for detection.”
Anomaly Detection for IoT Time-Series Data: A Survey (2020, methodology): “An LSTM-based encoder–decoder neural network is employed in [56] on a variety of univariate time series where the reconstruction error of the autoencoder is used to identify anomalous sequences within the data, their method is a semisupervised approach in that the initial network is trained only…”
Smart Anomaly Detection in Sensor Systems (2020, methodology): “[70] combine LSTM and autoencoder for multi sensor anomaly detection.”
Deep Learning for Time Series Anomaly Detection: A Survey (2022, neutral): “The EncDec-AD [124] model detects anomalies even from unpredictable univariate time series, in contrast to many existing models for anomaly detection.”
A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder (2017, methodology): “In particular, Malhorta et al. introduced an LSTM-based encoder-and-decoder (EncDec-AD) that estimates a reconstruction error [21].”
Multivariate Time-series Anomaly Detection via Graph Attention Network (2020, methodology): “al [3] proposes a LSTM-based encoder-decoder network that models reconstruction probabilities of the “normal” timeseries, and the reconstruction errors are utilized to detect anomalies from multiple sensors.”
Multivariate Time Series Anomaly Detection and Interpretation using Hierarchical Inter-Metric and Temporal Embedding (2021, methodology): “…has been an active research topic in the SIGKDD community these years [2, 14, 30], and is widely used to monitor the status ( i.e. , metrics) of entities ( e.g. , systems, services) in the application domain of manufacturing industry and Information Technology (IT) systems [14, 19, 20, 22, 30, 34].”
Deep Learning Based Forecasting of Critical Infrastructure Data (2017, methodology): “To apply a forecasting model to a non-stationary time series, it needs to be transformed into a stationary one [32].”