The base-rate fallacy and the difficulty of intrusion detection

By Stefan Axelsson. ACM Transactions on Privacy and Security, 2000

733 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Networks and Communications, Signal Processing

View Paper: https://www.semanticscholar.org/paper/d32865d2c97e7b216af94b1d7ef53fed80bf0667

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Signal Processing

Citations in the corpus

Network Anomaly Detection: Methods, Systems and Tools (2014, background): “1) Accuracy: Accuracy is a metric that measures how correctly an IDS works, measuring the percentage of detection and failure as well as the number of false alarms that the system produces [223], [224].”
Credit card fraud detection: A fusion approach using Dempster-Shafer theory and Bayesian learning (2009, background): “Axelsson [22] has pointed out that due to the base-rate fallacy problem, the factor limiting the performance of an intrusion detection system is not the ability to identify intrusive behavior correctly but its ability to minimize false alarms.”
A comprehensive survey on machine learning for networking: evolution, applications and research opportunities (2018, background): “For SSH traffic, it achieves 95.9%DR and 2.8% FPR on theDalhousie dataset, 97.2% DR and 0.8% FPR on the AMP dataset, and 82.9% DR and 0.5% FPR on the MAWI dataset.”
Experimental Study with Real-world Data for Android App Security Analysis using Machine Learning (2015, background): “In addition, PRC can reveal the e↵ect of the base-rate fallacy [7] in which while the overall percentages of benign apps incorrectly rejected is low, the count is still orders of magnitude greater than the count of those correctly rejected.”
Limiting the Impact of Stealthy Attacks on Industrial Control Systems (2016, background): “This quantity has a couple of advantages over the false alarm rate: (1) it addresses the deceptive nature of low false alarm rates due to the base-rate fallacy [5], and (2) it addresses the problem that several anomaly detection statistics make a decision (“alarm” or “normal behavior”) at non-consta”
A new approach to intrusion detection using Artificial Neural Networks and fuzzy clustering (2010, background): “The following measurements are often proposed to evaluate the detection precision of IDS (Axelsson, 2003): true positives, true negatives, false positives, and false negatives.”
Electricity Theft Detection in AMI Using Customers’ Consumption Patterns (2016, background): “As argued in [19], due to the base-rate fallacy phenomenon, the false alarm rate is the limiting factor for the performance of an intrusion detection system (IDS).”
A detailed analysis of the KDD CUP 99 data set (2009, methodology): “Besides, the performance measure applied in DARPA’98 Evaluation, ROC Curves, has been widely criticized, and since then many researchers have proposed new measures to overcome the existing deﬁciencies [8], [9], [10], [11], [12].”
TESSERACT: Eliminating Experimental Bias in Malware Classification across Space and Time (2018, background): “A common experimental bias in network intrusion detection is the base rate fallacy [5]: given the large imbalance between benign and malicious traffic, even FPR = 0.”
Credit Card Fraud Detection Using Hidden Markov Model (2012, unknown): “Otherwise,duetothe“baseratefallacy”effect[17], bankadministratorsmaytendtoignorethealarms.”
Anomaly-based network intrusion detection: Techniques, systems and challenges (2009, background): “Some of the most significant challenges in the area are: – Low detection efficiency, especially due to the high false positive rate usually obtained (Axelsson, 2000).”
Enhancing byte-level network intrusion detection signatures with context (2003, background): “[3] discusse s ome statistical limits, arguing in particular that the false-a larm rate is the limiting factor for the performance of an IDS.”
Hidden Markov Model (2010, unknown): “Otherwise, due to the “base rate fallacy” effect [17], bank administrators may tend to ignore the alarms.”
A Probabilistic Discriminative Model for Android Malware Detection with Decompiled Source Code (2015, background): “This probability may not be so meaningful under the situation of imbalanced data, due to the Base Rate Fallacy [31].”
Combating imbalance in network intrusion datasets (2006, background): “Axelsson [ 7 ] demonstrated that the primary limitation of an intrusion detection system is not the ability to identify behavior as intrusive, rather its effectiveness stems from its abilities to limit false alarms.”