Incremental Support Vector Learning for Ordinal Regression

By Bin Gu, Victor S. Sheng, KengYeow Tay, Walter Romano, Sijie Li. IEEE Transactions on Neural Networks and Learning Systems, 2015

712 citations recorded by the source corpus.

Support vector ordinal regression (SVOR) is a popular method to tackle ordinal regression problems. However, until now there were no effective algorithms proposed to address incremental SVOR learning due to the complicated formulations of SVOR. Recently, an interesting accurate on-line algorithm was proposed for training ν-support vector classification (ν-SVC), which can handle a quadratic formulation with a pair of equality constraints. In this paper, we first present a modified SVOR formulation based on a sum-of-margins strategy. The formulation has multiple constraints, and each constraint includes a mixture of an equality and an inequality. Then, we extend the accurate on-line ν-SVC algorithm to the modified formulation, and propose an effective incremental SVOR algorithm. The algorithm can handle a quadratic formulation with multiple constraints, where each constraint is constituted of an equality and an inequality. More importantly, it tackles the conflicts between the equality and inequality constraints. We also provide the finite convergence analysis for the algorithm. Numerical experiments on the several benchmark and real-world data sets show that the incremental algorithm can converge to the optimal solution in a finite number of steps, and is faster than the existing batch and incremental SVOR algorithms. Meanwhile, the modified formulation has better accuracy than the existing incremental SVOR algorithm, and is as accurate as the sum-of-margins based formulation of Shashua and Levin.

Subfields: Computational Theory and Mathematics, Computer Vision and Pattern Recognition, Management Science and Operations Research

View Paper: https://www.semanticscholar.org/paper/1608152248c3c5be6e7134351efe21390434207d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Computer Vision and Pattern Recognition

In Management Science and Operations Research

Citations in the corpus

A survey of depth and inertial sensor fusion for human action recognition (2015, methodology): “Classical classifiers such as support vector machine (SVM) [9, 29] and hidden Markov model (HMM) [23] are often employed for action recognition, e.g., [43, 47].”
Location Privacy Protection Based on Differential Privacy Strategy for Big Data in Industrial Internet of Things (2018, methodology): “Especially, the functional mechanism is introduced in [19], in which an objective function of the ε-differential privacy disturbance optimization problem is used to protect privacy.”
Incremental learning algorithms and applications (2016, background): “function to facilitate incrementality [141] and also possibly control complexity [58, 57].”
A novel intelligent diagnosis method using optimal LS-SVM with improved PSO algorithm (2017, methodology): “Gu et al. (2015) proposed an effective incremental support vec-torordinalregressionformulationbasedonasum-of-margins strategy.”
Data-driven health estimation and lifetime prediction of lithium-ion batteries: A review (2019, methodology): “In order to improve the stability and robustness of SVR with largescale training samples, decremental and incremental strategies [136,137] have been applied to integrate the relevant data sample for SVR training and ignore the irrelevant part.”
Firefly algorithm with neighborhood attraction (2017, methodology): “[25] presented a new FA for stock market price forecasting, which employs two new strategies, namely chaotic FA and support vector regression (SVR) [19].”
Disease Diagnosis in Smart Healthcare: Innovation, Technologies and Applications (2017, methodology): “For regression algorithms, typical examples are support vector regression [68], neural network [66], generalized linear models [69] and linear and non-linear regression [70].”
Social media big data analytics: A survey (2019, neutral): “3) Velocity : Most machine learning techniques rely on data availability, which implies that learning cannot start when the entire dataset is unavailable (Gu, Sheng, Tay, Romano, & Li, 2015).”
Ordinal Regression Methods: Survey and Experimental Study (2016, methodology): “Finally, an incremental version of SVOR algorithms is proposed in [97].”