The Role of Balanced Training and Testing Data Sets for Binary Classifiers in Bioinformatics

By Qiong Wei, Roland L. Dunbrack. PLoS ONE, 2013

297 citations recorded by the source corpus.

Training and testing of conventional machine learning models on binary classification problems depend on the proportions of the two outcomes in the relevant data sets. This may be especially important in practical terms when real-world applications of the classifier are either highly imbalanced or occur in unknown proportions. Intuitively, it may seem sensible to train machine learning models on data similar to the target data in terms of proportions of the two binary outcomes. However, we show that this is not the case using the example of prediction of deleterious and neutral phenotypes of human missense mutations in human genome data, for which the proportion of the binary outcome is unknown. Our results indicate that using balanced training data (50% neutral and 50% deleterious) results in the highest balanced accuracy (the average of True Positive Rate and True Negative Rate), Matthews correlation coefficient, and area under ROC curves, no matter what the proportions of the two phenotypes are in the testing data. Besides balancing the data by undersampling the majority class, other techniques in machine learning include oversampling the minority class, interpolating minority-class data points and various penalties for misclassifying the minority class. However, these techniques are not commonly used in either the missense phenotype prediction problem or in the prediction of disordered residues in proteins, where the imbalance problem is substantial. The appropriate approach depends on the amount of available data and the specific problem at hand.

Subfields: Artificial Intelligence, Genetics, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/97d667dbff26423037bc23240579a42d39e68aee

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mauno Vihinen
  2. Abhishek Niroula

In Genetics

  1. Abhishek Niroula
  2. Mauno Vihinen

In Molecular Biology

  1. Abhishek Niroula
  2. Mauno Vihinen

Citations in the corpus