Classifying Software Changes: Clean or Buggy?

By Sunghun Kim, E. James Jr. Whitehead, Yi Zhang. IEEE Transactions on Software Engineering, 2008

661 citations recorded by the source corpus.

This paper introduces a new technique for predicting latent software bugs, called change classification. Change classification uses a machine learning classifier to determine whether a new software change is more similar to prior buggy changes or clean changes. In this manner, change classification predicts the existence of bugs in software changes. The classifier is trained using features (in the machine learning sense) extracted from the revision history of a software project stored in its software configuration management repository. The trained classifier can classify changes as buggy or clean, with a 78 percent accuracy and a 60 percent buggy change recall on average. Change classification has several desirable qualities: 1) The prediction granularity is small (a change to a single file), 2) predictions do not require semantic information about the source code, 3) the technique works for a broad array of project types and programming languages, and 4) predictions can be made immediately upon the completion of a change. Contributions of this paper include a description of the change classification approach, techniques for extracting features from the source code and change histories, a characterization of the performance of change classification across 12 open source projects, and an evaluation of the predictive power of different groups of features.

Subfields: Computer Networks and Communications, Information Systems, Software

View Paper: https://www.semanticscholar.org/paper/9f905c4050131f82bcfb503a580cfcfdedc25fc9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

In Software

Citations in the corpus

RCLinker: Automated Linking of Issue Reports and Commits Leveraging Rich Contextual Information (2015, positive): “Furthermore, these links can be used to generate high quality defect data (i.e., the number of defects that affect various classes in a project during its life-time), which can, in turn, be used to build effective bug prediction solutions [24], [7], [43], [28], [41], [63], [21].”
Dealing with noise in defect prediction (2011, methodology): “Detailed feature extraction methods of CC can be found in [11].”
It's not a bug, it's a feature: How misclassification impacts bug prediction (2013, methodology): “[12] uses version control history to verify that applied code changes are actual fixes.”
Predicting Vulnerable Software Components via Text Mining (2014, methodology): “Kim et al. [28] have gone even further by using certain change metadata, complexity metrics, change log metrics and filenames in addition to the change source code to build a prediction model.”
The impact of code review coverage and code review participation on software quality: a case study of the qt, VTK, and ITK projects (2014, methodology): “A similar approach was used to determine defect-fixing and defect-inducing changes in other work [18] [20].”
The future of employment: How susceptible are jobs to computerisation? (2017, background): “Algorithms can further automatically detect bugs in software (Hangal and Lam, 2002; Livshits and Zimmermann, 2005; Kim et al., 2008), with a reliability that humans are unlikely to match.”
Deep Learning for Just-in-Time Defect Prediction (2015, methodology): “We use cost effectiveness and F1-score which are also used by past software engineering studies to evaluate the effectiveness of various prediction techniques [7], [12], [16], [33], [34], [17], [14], [13], [35].”
An empirical study of the impact of modern code review practices on software quality (2015, methodology): “A similar approach was used to determine defect-fixing and defectinducing changes in other work (Mockus and Votta, 2000; Hassan, 2008; Kim et al., 2008; Kamei et al., 2013).”
Easy over hard: a case study on deep learning (2017, methodology): “In this work, we use F1 to score the candidate parameters because it controls the trade-o between precision and recall, which is also consistent with XU [74] and is also widely used in soware engineering community to evaluate classication results [21, 33, 46, 68].”
The impact of tangled code changes (2013, methodology): “[12] developed a change classification technique classify changes as “buggy” or “clean” with a precision of 75% and a recall of 65% (on average).”
Transfer defect learning (2013, methodology): “In this paper, we apply a state-of-the-art transfer learning approach, TCA, to make feature distributions in source and target projects similar.”
A systematic review of machine learning techniques for software fault prediction (2015, background): “Hence, total number of 122 studies [1, 2, 20-139] were identified for further processing and analysis.”
A systematic and comprehensive investigation of methods to build and evaluate fault prediction models (2010, methodology): “(Kim et al., 2008; Pai and Dugan, 2007; Denaro and Pezze, 2002), while other studies achieved recall close to 1 (Elish and Elish, 2008).”
Learning to rank relevant files for bug reports using domain knowledge (2014, background): “Moser et al. [30] and Kim et al. [20] use machine learning techniques to train a prediction model based on code changes.”
A large-scale empirical study of just-in-time quality assurance (2013, methodology): “To build a change risk model, we use a wide range of factors based on the characteristics of a software change, such as the number of added lines, and developer experience.”