Robust prediction of fault-proneness by random forests

By Lan Guo, Yan Ma, Bojan Cukic, Harshinder Singh. IEEE International Symposium on Software Reliability Engineering, 2004

322 citations recorded by the source corpus.

Accurate prediction of fault prone modules (a module is equivalent to a C function or a C+ + method) in software development process enables effective detection and identification of defects. Such prediction models are especially beneficial for large-scale systems, where verification experts need to focus their attention and resources to problem areas in the system under development. This paper presents a novel methodology for predicting fault prone modules, based on random forests. Random forests are an extension of decision tree learning. Instead of generating one decision tree, this methodology generates hundreds or even thousands of trees using subsets of the training data. Classification decision is obtained by voting. We applied random forests in five case studies based on NASA data sets. The prediction accuracy of the proposed methodology is generally higher than that achieved by logistic regression, discriminant analysis and the algorithms in two machine learning software packages, WEKA [I. H. Witten et al. (1999)] and See5. The difference in the performance of the proposed methodology over other methods is statistically significant. Further, the classification accuracy of random forests is more significant over other methods in larger data sets.

Subfields: Information Systems, Software

View Paper: https://www.semanticscholar.org/paper/d05d30143e8ef392c0df86b20e039953016eb6c1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

In Software

Citations in the corpus

Anomaly Based Network Intrusion Detection with Unsupervised Outlier Detection (2006, methodology): “For instance, it has been applied to prediction [16, 17], probability estimation [24], and pattern analysis in multimedia information retrieval and bioinformatics [18].”
A systematic and comprehensive investigation of methods to build and evaluate fault prediction models (2010, methodology): “Guo et al. (2004) compared 27 modeling techniques including logistic regression and 20 techniques available through the WEKA tool.”
The Impact of Automated Parameter Optimization on Defect Prediction Models (2018, unknown): “dom forest classification technique in defect prediction studies [36], [39], [42], [44], [52], [80], [116], [137], [149].”
Predicting the location and number of faults in large software systems (2005, background): “Results along these lines can be found in [1], [2], [3], [4], [5], [6], [7], [10], [11], [12], [15], among others.”
Predicting defect-prone software modules using support vector machines (2008, methodology): “(Guo et al., 2004) proposed random forest technique to predict fault‐proneness of software system.”
A General Software Defect-Proneness Prediction Framework (2011, methodology): “The third type of work classifies software components as defect-prone and non-defect-prone by means of metricbased classification [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24].”
Benchmarking Classification Models for Software Defect Prediction: A Proposed Framework and Novel Findings (2008, methodology): “…by means of their TP rate (TPR), also known as sensitivity, rate of detection, or hit rate, and by their FP rate (FPR) or false alarm rate (e.g., [24], [32], [44], [67]): We argue that such error-based metrics, although having undoubted practical value, are conceptually inappropriate for…”
Random-Forests-Based Network Intrusion Detection Systems (2008, methodology): “For instance, it has been applied to prediction [ 17 ], [25] and probability estimation [31].”
An Empirical Comparison of Model Validation Techniques for Defect Prediction Models (2017, background): “…validation technique is the number of Events Per Variable [2], [6], [82], [99], i.e., the ratio of the number of occurrences of the least frequently occurring class of the dependent variable (i.e., the events) to the number of independent variables used to train the model (i.e., the variables).”