Fair and balanced?: bias in bug-fix datasets

By Christian Bird, Adrian Bachmann, Eirik Aune, John Duffy, Abraham Bernstein, Vladimir Filkov, Premkumar T. Devanbu. ESEC/SIGSOFT FSE, 2009

385 citations recorded by the source corpus.

Subfields: Computer Networks and Communications, Information Systems, Software

View Paper: https://www.semanticscholar.org/paper/acaec056144f5ba306287b06027e2a621651f977

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

In Software

Citations in the corpus

An extensive comparison of bug prediction approaches (2010, background): “Bird et al. studied this problem in bug databases [ 27 ]: They observed that the set of bugs which are linked to commit comments is not a fair representation of the full population of bugs.”
The Impact of Automated Parameter Optimization on Defect Prediction Models (2018, background): “For example, concerns about the quality of defect datasets have been raised [13], [102], [114], [137], [148].”
The promises and perils of mining GitHub (2009, background): “[5] found that this set of bugs is a biased sample of the entire population.”
Dealing with noise in defect prediction (2011, methodology): “In addition, after eliminating the noises using our algorithm, defect prediction accuracy is improved.”
History Driven Program Repair (2016, background): “This is a difficult problem in repository mining [8].”
What Makes a Good Bug Report? (2008, background): “investigated the effects of bias in bug data sets for open-source projects [17].”
Cross-project defect prediction: a large scale experiment on data vs. domain vs. process (2009, background): “studied bias in bug datasets and found that prior experience as well as severity can influence how well bugs are linked to version archives [5].”
Researcher Bias: The Use of Machine Learning in Software Defect Prediction (2014, background): “, NASA and Eclipse [12], [52] and this might jeopardise the”
ReLink: recovering links between bugs and changes (2011, methodology): “Bird et al. found that the BugCache algorithm [17] is sensitive to the biased data [7].”
Information needs for software development analytics (2012, background): “To make decisions based on data, the quality of the data has to be high [41], [42], [43].”
The impact of tangled code changes (2013, background): “The effects of bias caused by unbalanced data sets on defect prediction models were investigated by various studies [7], [8], [9].”
Evaluating defect prediction approaches: a benchmark and an extensive comparison (2011, background): “Bird et al. studied this problem in bug databases ( Bird et al. 2009 ): They observed that the set of bugs which are linked to commit comments is not a fair representation of the full population of bugs.”
It's not a bug, it's a feature: How misclassification impacts bug prediction (2013, result): “Recent studies investigates bug and commit bias and their potential implication for defect prediction models which might interfere with the results presented in this paper [31], [32], [33].”
An in-depth study of the promises and perils of mining GitHub (2015, background): “Furthermore, (Bird et al. 2009b) described both the promises and perils associated with exploiting the information stored in git, a decentralized version control system.”
A Similarity Measure for Text Classification and Clustering (2014, methodology): “The live is applied in many text applications, together with single label classification, multi-label classification and hierarchal agglomerate cluster, and also the results obtained demonstrate the effectiveness of the projected similarity live [15].”
Transfer defect learning (2013, methodology): “We use these outcomes to evaluate the accuracy of defect prediction with the measures defined below [21], [23], [41]: Precision: The ratio of the number of instances correctly classified as buggy (Nb→b) to the number of instances classified as buggy.”
RCLinker: Automated Linking of Issue Reports and Commits Leveraging Rich Contextual Information (2015, neutral): “Unfortunately, many issue reports can not be linked to their corresponding commits for many reasons [4], [3], [49].”