Handling concept drifts in incremental learning with support vector machines

By N. Syed, Huan Liu, Kah Kay Sung. 1999

With the increase in the size of real-world databases, there is an ever-increasing need to scale up inductive learning algorithms. Incremental learning techniques are one possible solution to the scalability problem. In this paper, we propose three ctiteria to evaluate the robustness and reliability of incremental learning methods, and use them to study the robustness of an incremental training method for Support Vector Machines. We provide empirical results using benchmark machine learning datasets to show that support vectors form a svccdnct and suficient set for block-by-block incremental learning.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Distribution-Free One-Pass Learning (2017, methodology): “Base classifiers are 1NN and SVM, denoted as 1NN-win and SVM-win [20]; (c) SVM-fix, batch implementation of SVM with a fixed window size [34]; (d) SVM-ada, batch implementation of SVM with an adaptive window size [18]; (e) DWM, dynamic weighted majority algorithm, an adaptive ensemble based on the t”
A survey on concept drift adaptation (2014, background): “[Salganicoff 1997], [Widmer and Kubat 1996], Fixed Size [Syed et al. 1999], [Hulten et al.”
Applying lazy learning algorithms to tackle concept drift in spam filtering (2007, background): “Instance weighting uses the ability of some learning algorithms such as support vector machines to process weighted instances (Klinkenberg & Joachims, 2000; Syed, Liu, & Sung, 1999; Taylor, Nakhaeizadeh, & Lanquillon, 1997).”
Detecting Concept Drift with Support Vector Machines (2000, background): “Syed et al. (1999) describe an approach to incrementally learning support vector machines that handles virtual concept drift implied by incrementally learning from several subsamples of a large training set, but they do not address the problem of (real) concept drift addressed here.”
Learning under Concept Drift: an Overview (2010, background): “Past support vectors are transfered and combined with the recent training data in [145].”
An incremental face recognition system based on deep learning (2017, background): “Syed et al.[11] showed that the decision function of an SVM depends only on its support vectors.”
Dynamic Weighted Majority: An Ensemble Method for Drifting Concepts (2007, methodology): “The term concept drift has been applied to different phenomena, such as drops and recoveries in performance during online learning (Syed et al., 1999).”
Online Machine Learning in Big Data Streams (2018, methodology): “More examples include Bayesian models [94, 19], neural networks [95, 153], and SVM [215, 137].”
Learning drifting concepts: Example selection vs. example weighting (2004, unknown): “Of course this may again lead to a loss of accuracy in the results as it does in the case without (real) concept drift [26,29].”
Incremental learning with support vector machines (2001, methodology): “The problem of drifting concepts in incremental Support Vector Machine learning has been addressed in [Syed et al., 1999a] where it has been experimentally validated that SVMs handle drifting concepts well with respect to the criteria of stability of the result during the learning steps,…”
On Issues of Instance Selection (2002, background): “In Syed et al. (1999b) , for example, they investigated if support vector machines could be used as a general model for instance selection in classification.”
On-line Boosting and Vision (2006, background): “A closely related topic is concept drift [122, 127].”