Real-time big data processing for anomaly detection: A Survey

By R. Habeeb, Fariza Hanum Binti Md Nasaruddin, A. Gani, Ibrahim Abaker Targio Hashem, Ejaz Ahmed, Muhammad Imran. International Journal of Information Management, 2019

438 citations recorded by the source corpus.

Abstract The advent of connected devices and omnipresence of Internet have paved way for intruders to attack networks, which leads to cyber-attack, financial loss, information theft in healthcare, and cyber war. Hence, network security analytics has become an important area of concern and has gained intensive attention among researchers, off late, specifically in the domain of anomaly detection in network, which is considered crucial for network security. However, preliminary investigations have revealed that the existing approaches to detect anomalies in network are not effective enough, particularly to detect them in real time. The reason for the inefficacy of current approaches is mainly due the amassment of massive volumes of data though the connected devices. Therefore, it is crucial to propose a framework that effectively handles real time big data processing and detect anomalies in networks. In this regard, this paper attempts to address the issue of detecting anomalies in real time. Respectively, this paper has surveyed the state-of-the-art real-time big data processing technologies related to anomaly detection and the vital characteristics of associated machine learning algorithms. This paper begins with the explanation of essential contexts and taxonomy of real-time big data processing, anomalous detection, and machine learning algorithms, followed by the review of big data processing technologies. Finally, the identified research challenges of real-time big data processing in anomaly detection are discussed.

Subfields: Artificial Intelligence, Computer Networks and Communications, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/3cfe075af77bf0364e6ddecb3d223960d06e8927

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Isaac Triguero
2. Enrique Onieva
3. Mikel Canizo
4. Ali Bou Nassif
5. Angel Conde

In Computer Networks and Communications

1. Ali Bou Nassif
2. Qassim Nasir

In Control and Systems Engineering

1. Isaac Triguero
2. Enrique Onieva
3. Mikel Canizo
4. Angel Conde

Citations in the corpus

• Multi-head CNN-RNN for multi-time series anomaly detection: An industrial case study (2019, background): “Time series anomaly detection is a very relevant field in computer science and data mining [1, 2, 3].”
• Prescriptive analytics: Literature review and research challenges (2020, neutral): “Currently, the vast majority of business analytics efforts are spent on descriptive analytics and predictive analytics with typical methodologies including data mining, machine learning, artificial intelligence and simulation (den Hertog & Postek, 2016; Habeeb et al., 2018; Larose & Larose, 2015).”
• Machine Learning for Anomaly Detection: A Systematic Review (2021, background): “Anomalies are classified into three main categories [1], [9], [10]: 1.”
• The Role of Big Data Analytics in Industrial Internet of Things (2019, neutral): “MES helps keep the track record of all manufacturing information in real time and receive up-to-date data from robots, machine and IoT devices [79].”