Deep Learning for Unsupervised Insider Threat Detection in Structured Cybersecurity Data Streams

By Aaron Tuor, Samuel Kaplan, Brian Hutchinson, Nicole Nichols, Sean Robinson. AAAI Workshops, 2017

365 citations recorded by the source corpus.

Analysis of an organization's computer network activity is a key component of early detection and mitigation of insider threat, a growing concern for many organizations. Raw system logs are a prototypical example of streaming data that can quickly scale beyond the cognitive power of a human analyst. As a prospective filter for the human analyst, we present an online unsupervised deep learning approach to detect anomalous network activity from system logs in real time. Our models decompose anomaly scores into the contributions of individual user behavior features for increased interpretability to aid analysts reviewing potential cases of insider threat. Using the CERT Insider Threat Dataset v6.2 and threat detection recall as our performance metric, our novel deep and recurrent neural network models outperform Principal Component Analysis, Support Vector Machine and Isolation Forest based anomaly detection baselines. For our best model, the events labeled as insider threat activity in our dataset had an average anomaly score in the 95.53 percentile, demonstrating our approach's potential to greatly reduce analyst workloads.

Subfields: Computer Networks and Communications, Information Systems

View Paper: https://www.semanticscholar.org/paper/bac292286c7d12df6863c7ca8dec720ed6288302

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

Citations in the corpus

Deep Learning for Anomaly Detection: A Survey (2019, methodology): “Variants of Unsupervised DAD models (Tuor et al. [2017]) are shown to outperform traditional methods such as principal component analysis (PCA) (Wold et al. [1987]), support vector machine (SVM) Cortes and Vapnik [1995] and Isolation Forest (Liu et al. [2008]) techniques in applications domains…”
Log2vec: A Heterogeneous Graph Embedding Based Approach for Detecting Cyber Threats within Enterprise (2019, methodology): “Analysts can set pro according to the existing work and specific scenarios [17, 34, 46, 57].”
An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series (2021, background): “In the unsupervised context, four studies [9]–[12] mention that ranking of scores or errors can be used to diagnose the cause of anomalies but only [10] shows experimental results on an open dataset.”
Unsupervised Machine Learning for Networking: Techniques, Applications and Research Challenges (2017, methodology): “[33] RNN RNN and DNN are employed to extract feature from raw data which then used for threat assessment and insider threat detection in data streams.”
A Survey of Deep Learning Methods for Cyber Security (2019, methodology): “[167] used an unsupervised DL network on system log data for filtering out normal data.”
N-BaIoT—Network-Based Detection of IoT Botnet Attacks Using Deep Autoencoders (2018, methodology): “We empirically show that the autoencoders’ false alarm rate is considerably lower than three other algorithms commonly used for anomaly detection [13].”
Optimizing Space-Air-Ground Integrated Networks by Artificial Intelligence (2018, background): “In [14], the authors proposed a deep learning based approach to detect anomalous network activity.”
Ten Challenges in Advancing Machine Learning Technologies toward 6G (2020, background): “Even though deep learning has demonstrated its efﬁciency in threat detection [10], future communication services will require context-aware security protection with diversiﬁed data generated in people’s life and work.”