SMOTE: Synthetic Minority Over-sampling Technique
By Nitesh V. Chawla, Kevin W. Bowyer, L. Hall, W. Philip Kegelmeyer Jr.. 2002
31,281 citations recorded by the source corpus.
An approach to the construction of classifiers from imbalanced datasets is described. A dataset is imbalanced if the classification categories are not approximately equally represented. Often real-world data sets are predominately composed of "normal" examples with only a small percentage of "abnormal" or "interesting" examples. It is also the case that the cost of misclassifying an abnormal (interesting) example as a normal example is often much higher than the cost of the reverse error. Under-sampling of the majority (normal) class has been proposed as a good means of increasing the sensitivity of a classifier to the minority class. This paper shows that a combination of our method of oversampling the minority (abnormal)cla ss and under-sampling the majority (normal) class can achieve better classifier performance (in ROC space)tha n only under-sampling the majority class. This paper also shows that a combination of our method of over-sampling the minority class and under-sampling the majority class can achieve better classifier performance (in ROC space)t han varying the loss ratios in Ripper or class priors in Naive Bayes. Our method of over-sampling the minority class involves creating synthetic minority class examples. Experiments are performed using C4.5, Ripper and a Naive Bayes classifier. The method is evaluated using the area under the Receiver Operating Characteristic curve (AUC)and the ROC convex hull strategy.
Subfields: Artificial Intelligence, Optimization & Training, Statistical Learning Theory
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Mohit Bansal
- Quoc V. Le
- Sergey Levine
- Percy Liang
- Jianfeng Gao
- Dacheng Tao
- Trevor Darrell
In Optimization & Training
- Jiashi Feng
- Frank Hutter
- Yoshua Bengio
- Francisco Herrera
- Yann Dauphin
- Quoc V. Le
- Nitesh V. Chawla
- Alan L. Yuille
In Statistical Learning Theory
- Nitesh V. Chawla
- Francisco Herrera
- Taghi M. Khoshgoftaar
- Gary M. Weiss
- Alberto Fernández
- Nathalie Japkowicz
- Xin Yao
- Jason Van Hulse
Citations in the corpus
- mixup: Beyond Empirical Risk Minimization (2017, methodology): “15 mix inputs only SC + KNN (Chawla et al., 2002) 3 7 5.”
- AutoML: A Survey of the State-of-the-Art (2019, methodology): “To solve this problem, Synthetic Minority Over-Sampling Technique (SMOTE) [70] was proposed to synthesises new minority samples between existing real minority samples instead of up-sampling them or down-sampling the majority samples.”
- Differential Privacy Has Disparate Impact on Model Accuracy (2019, background): “Other approaches to balancing accuracy across classes include oversampling [8], adversarial training [3] with a loss function that overweights the underrepresented group, cost-sensitive learning [9], and re-sampling [6].”
- Data preprocessing techniques for classification without discrimination (2011, background): “[7], a synthetic minority over-sampling technique (SMOTE) for two-class problems that over-sampled the minority class by creating synthetic examples rather than replicating examples is proposed.”
- A Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective (2018, background): “The basic SMOTE algorithm finds for each minority example the k minority class nearest neighbors and then generates synthetic examples along the line segments joining the minority example and its nearest neighbors.”
- Understanding Data Augmentation for Classification: When to Warp? (2016, background): “In SMOTE the synthetic examples are created in feature-space from randomly selected pairs of real world feature-examples from the minority class.”
- Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims (2020, background): “Training data can be rigorously evaluated for representativeness [63] [64]; assumptions can be characterized by evaluating modular components of an AI model and by clearly communicating output uncertainties; and performance can be characterized by measuring generalization, fairness, and performance ”
- Box drawings for learning with imbalanced data (2014, methodology): “…classification, specifically kernel and active learning methods (Raskutti and Kowalczyk 2004; Wu and Chang 2003), and work on sampling (Abe 2003; Chawla et al. 2002) that includes un-dersampling, oversampling, and data generation, which can be used in conjunction with methods like the ones…”
- Using Pre-Training Can Improve Model Robustness and Uncertainty (2019, methodology): “Generating synthetic samples through linear interpolation between data samples belonging in the same minority class has been studied in Chawla et al. (2002).”
- Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learning (2022, background): “, B), achieving the calibration of the classifier, which is the simplest way adopted in the long-tail recognition [2],”
- Imbalanced Continual Learning with Partitioning Reservoir Sampling (2020, background): “More aggressively, there have also been directions that populate the minority samples via generation to avoid overfitting in the minority [14,58,23].”
- Natural Language Processing for EHR-Based Computational Phenotyping (2018, methodology): “cal analysis and machine learning have provided alternative ways to automatically generate phenotypes, high dimensional feature spaces, data sparsity, and data imbalance remain impediments to the adoption of these methods [194].”
- Learning to Compose Domain-Specific Transformations for Data Augmentation (2017, methodology): “For example, the well-known SMOTE algorithm applies this basic technique for oversampling in class-imbalanced settings [2], and recent work explores using a similar interpolation approach in a learned feature space [5].”
- Data Augmentation by Pairing Samples for Images Classification (2018, methodology): “The basic idea of our technique is simple: We synthesize a new image from two images randomly picked from the training set as done in some existing techniques, such as Chawla et al. (2002) and Wang & Perez (2017). To synthesize a new image, we take the most naive approach, just averaging the intensi”
- Decoupling Representation and Classifier for Long-Tailed Recognition (2019, methodology): “Various re-sampling strategies (Chawla et al., 2002; Shen et al., 2016; Cao et al., 2019), loss reweighting and margin regularization over few-shot classes are thus proposed.”
- Class-Balanced Loss Based on Effective Number of Samples (2019, background): “To solve this, novel samples can be either interpolated from neighboring samples [5] or synthesized [14, 51] for minor classes.”