Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric

By Sabri Boughorbel, Fethi Jarray, Mohammed El-Anbari. PLoS ONE, 2017

1,091 citations recorded by the source corpus.

Data imbalance is frequently encountered in biomedical applications. Resampling techniques can be used in binary classification to tackle this issue. However such solutions are not desired when the number of samples in the small class is limited. Moreover the use of inadequate performance metrics, such as accuracy, lead to poor generalization results because the classifiers tend to predict the largest size class. One of the good approaches to deal with this issue is to optimize performance metrics that are designed to handle data imbalance. Matthews Correlation Coefficient (MCC) is widely used in Bioinformatics as a performance metric. We are interested in developing a new classifier based on the MCC metric to handle imbalanced data. We derive an optimal Bayes classifier for the MCC metric using an approach based on Frechet derivative. We show that the proposed algorithm has the nice theoretical property of consistency. Using simulated data, we verify the correctness of our optimality result by searching in the space of all possible binary classifiers. The proposed classifier is evaluated on 64 datasets from a wide range data imbalance. We compare both classification performance and CPU efficiency for three classifiers: 1) the proposed algorithm (MCC-classifier), the Bayes classifier with a default threshold (MCC-base) and imbalanced SVM (SVM-imba). The experimental evaluation shows that MCC-classifier has a close performance to SVM-imba while being simpler and more efficient.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/8beb6c5c1968ac9e986fd69995a0bdeca1936099

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

On evaluation metrics for medical applications of artificial intelligence (2021, background): “PPV = # correct positive predictions # samples classified as positive = TP TP+FP (8)”
Classification assessment methods (2020, background): “A coefficient of þ 1 indicates a perfect prediction, − 1 represents total disagreement between prediction and true values and zero means that no better than random prediction [16,3].”
Evaluation metrics and statistical tests for machine learning (2024, methodology): “· = = · = = Similarly, MCC can be computed from a general k × k confusion matrix with the formula 27 In the special case k = 2 , we obtain the same formulas for Cohen’s κ and MCC as in (2) and (3) 22 .”
Error Metrics and Performance Fitness Indicators for Artificial Intelligence and Machine Learning in Engineering and Sciences (2020, background): “• Considered as a balanced measures as it involves values of all the four quardants of a confusion matrix [100].”
Cyberbullying Detection With Fairness Constraints (2020, methodology): “As accuracy, area under the receiver operating characteristic curve, and F1 score may lead to misleading conclusions on imbalanced datasets (such as here), we evaluate our models with Matthews correlation coefficient (MCC) which does not exhibit such drawback [60], [61], [62], [63].”
Metrics for Multi-Class Classification: an Overview (2020, methodology): “Macro-Average methods tend to calculate an overall mean of different measures, because the numerators of Macro Average Precision and Macro Average Recall are composed by values in the range [0, 1].”
The Matthews Correlation Coefficient (MCC) is More Informative Than Cohen’s Kappa and Brier Score in Binary Classification Assessment (2021, background): “Recently, Boughorbel and colleagues [13] described an enhanced classifier based on the Matthews correlation coefficient, while Zhu [14] investigated the behavior of MCC on several imbalanced cases.”