AdaBoost with SVM-based component classifiers

By Xuchun Li, Lei Wang, Eric Sung. Engineering applications of artificial intelligence, 2008

429 citations recorded by the source corpus.

The use of SVM (Support Vector Machine) as component classifier in AdaBoost may seem like going against the grain of the Boosting principle since SVM is not an easy classifier to train. Moreover, Wickramaratna et al. [2001. Performance degradation in boosting. In: Proceedings of the Second International Workshop on Multiple Classifier Systems, pp. 11-21] show that AdaBoost with strong component classifiers is not viable. In this paper, we shall show that AdaBoost incorporating properly designed RBFSVM (SVM with the RBF kernel) component classifiers, which we call AdaBoostSVM, can perform as well as SVM. Furthermore, the proposed AdaBoostSVM demonstrates better generalization performance than SVM on imbalanced classification problems. The key idea of AdaBoostSVM is that for the sequence of trained RBFSVM component classifiers, starting with large @s values (implying weak learning), the @s values are reduced progressively as the Boosting iteration proceeds. This effectively produces a set of RBFSVM component classifiers whose model parameters are adaptively different manifesting in better generalization as compared to AdaBoost approach with SVM component classifiers using a fixed (optimal) @s value. From benchmark data sets, we show that our AdaBoostSVM approach outperforms other AdaBoost approaches using component classifiers such as Decision Trees and Neural Networks. AdaBoostSVM can be seen as a proof of concept of the idea proposed in Valentini and Dietterich [2004. Bias-variance analysis of support vector machines for the development of SVM-based ensemble methods. Journal of Machine Learning Research 5, 725-775] that Adaboost with heterogeneous SVMs could work well. Moreover, we extend AdaBoostSVM to the Diverse AdaBoostSVM to address the reported accuracy/diversity dilemma of the original Adaboost. By designing parameter adjusting strategies, the distributions of accuracy and diversity over RBFSVM component classifiers are tuned to maintain a good balance between them and promising results have been obtained on benchmark data sets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/5e9e028c20a3f4c9383e02ddecf85e0d03b2f174

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Multifractal Texture Estimation for Detection and Segmentation of Brain Tumors (2013, methodology): “The authors show that Diverse AdaBoostSVM offers superior performance over its counterparts for unbalanced dataset.”
Cost-sensitive learning methods for imbalanced data (2010, methodology): “Most of them focus on the manipulation at the data level (resampling methods) such as in [4], [5], [6], [7], [8], [9], [10] and the classifier level (changing the classifier internally) such as in [11], [12], [13], [14], [15], [16], [17], [18].”
Adaptive Machine Learning for Credit Card Fraud Detection (2015, methodology): “[110] use SVM with RBF kernels as base classifier for AdaBoost.”
DUNet: A deformable network for retinal vessel segmentation (2018, background): “Various classiﬁers such as k-NN classiﬁer [19], support vector machine (SVM) [20] [21], artiﬁcial neural networks (ANN) [22], AdaBoost [23] etc, have been proposed to deal with different tasks.”