NetFlow Datasets for Machine Learning-based Network Intrusion Detection Systems

By Mohanad Sarhan, Siamak Layeghy, Nour Moustafa, Marius Portmann. BDTA/WiCON, 2020

325 citations recorded by the source corpus.

Machine Learning (ML)-based Network Intrusion Detection Systems (NIDSs) have proven to become a reliable intelligence tool to protect networks against cyberattacks. Network data features has a great impact on the performances of ML-based NIDSs. However, evaluating ML models often are not reliable, as each ML-enabled NIDS is trained and validated using different data features that may do not contain security events. Therefore, a common ground feature set from multiple datasets is required to evaluate an ML model's detection accuracy and its ability to generalise across datasets. This paper presents NetFlow features from four benchmark NIDS datasets known as UNSW-NB15, BoT-IoT, ToN-IoT, and CSE-CIC-IDS2018 using their publicly available packet capture files. In a real-world scenario, NetFlow features are relatively easier to extract from network traffic compared to the complex features used in the original datasets, as they are usually extracted from packet headers. The generated Netflow datasets have been labelled for solving binary- and multiclass-based learning challenges. Preliminary results indicate that NetFlow features lead to similar binary-class results and lower multi-class classification results amongst the four datasets compared to their respective original features datasets. The NetFlow datasets are named NF-UNSW-NB15, NF-BoT-IoT, NF-ToN-IoT, NF-CSE-CIC-IDS2018 and NF-UQ-NIDS are published at this http URL for research purposes.

Subfields: Artificial Intelligence, Computer Networks and Communications, Hardware and Architecture

View Paper: https://www.semanticscholar.org/paper/9c4f0d030ea394bfd5be248b7bafbcad92e83080

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Marius Portmann
2. Mohanad Sarhan
3. Siamak Layeghy

In Computer Networks and Communications

1. Marius Portmann
2. Mohanad Sarhan
3. Siamak Layeghy

Citations in the corpus

• Towards a Standard Feature Set for Network Intrusion Detection System Datasets (2021, methodology): “A network data flow is a sequence of packets, in either uni- or bi-direction, between two unique endpoints sharing some attributes such as source/destination IP address and L4 (transport layer) ports, and the L4 protocol, also known as the five-tuple [11].”