SMOTE for high-dimensional class-imbalanced data

By Rok Blagus, Lara Lusa. BMC Bioinformatics, 2013

1,106 citations recorded by the source corpus.

Classification using class-imbalanced data is biased in favor of the majority class. The bias is even larger for high-dimensional data, where the number of variables greatly exceeds the number of samples. The problem can be attenuated by undersampling or oversampling, which produce class-balanced data. Generally undersampling is helpful, while random oversampling is not. Synthetic Minority Oversampling TEchnique (SMOTE) is a very popular oversampling method that was proposed to improve random oversampling but its behavior on high-dimensional data has not been thoroughly investigated. In this paper we investigate the properties of SMOTE from a theoretical and empirical point of view, using simulated and real high-dimensional data. While in most cases SMOTE seems beneficial with low-dimensional data, it does not attenuate the bias towards the classification in the majority class for most classifiers when data are high-dimensional, and it is less effective than random undersampling. SMOTE is beneficial for k-NN classifiers for high-dimensional data if the number of variables is reduced performing some type of variable selection; we explain why, otherwise, the k-NN classification is biased towards the minority class. Furthermore, we show that on high-dimensional data SMOTE does not change the class-specific mean values while it decreases the data variability and it introduces correlation between samples. We explain how our findings impact the class-prediction for high-dimensional data. In practice, in the high-dimensional setting only k-NN classifiers based on the Euclidean distance seem to benefit substantially from the use of SMOTE, provided that variable selection is performed before using SMOTE; the benefit is larger if more neighbors are used. SMOTE for k-NN without variable selection should not be used, because it strongly biases the classification towards the minority class.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/e2e3bdc7f4df4685fce1b3c51a303056ff7f9851

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistics and Probability

Citations in the corpus

An empirical survey of data augmentation for time series classification with neural networks (2020, methodology): “SMOTE has been shown to perform well in many time series applications, such as sensor data [44], gene sequences [63], high-dimensional data [30].”
Deep Learning Based Vulnerability Detection: Are We There Yet? (2020, background): “SMOTE has shown to be effective in a number of domains with imbalanced datasets [37], [38].”
SMOTE for Learning from Imbalanced Data: Progress and Challenges, Marking the 15-year Anniversary (2018, methodology): “In addition, since SMOTE applies an interpolation procedure to generate new synthetic data on the feature space, we will analyze the curse of dimensionality (Blagus & Lusa, 2013) as well as different aspects for the interpolation process (Section 5.4).”
The Impact of Class Rebalancing Techniques on the Performance and Interpretation of Defect Prediction Models (2018, background): “[86]’s findings which points out that SMOTE does not perform well with high-dimensionality data.”
Industrial Artificial Intelligence in Industry 4.0 - Systematic Review, Challenges and Outlook (2020, background): “While SMOTE and its extensions have achieved considerable results in recent years [79], studies have shown that for high-dimensionality problems its effects are underwhelming [80].”