Supervised classification and mathematical optimization

By Emilio Carrizosa, Dolores Romero Morales. 2013

159 citations recorded by the source corpus.

Data mining techniques often ask for the resolution of optimization problems. Supervised classification, and, in particular, support vector machines, can be seen as a paradigmatic instance. In this paper, some links between mathematical optimization methods and supervised classification are emphasized. It is shown that many different areas of mathematical optimization play a central role in off-the-shelf supervised classification methods. Moreover, mathematical optimization turns out to be extremely useful to address important issues in classification, such as identifying relevant variables, improving the interpretability of classifiers or dealing with vagueness/noise in the data.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Methods and Models for Interpretable Linear Classification (2014, methodology): “Some of these methods aim to can extract simple transparent models from black-box models such as neural networks [92], random forests [67, 95], and support vector machines [30, 65, 15].”
Optimal randomized classification trees (2021, methodology): “The methodology in oblique.tree is also a greedy approach that is based on Mathematical Optimization: the oblique cuts are obtained as the output of a Logistic Regression classifier.”
A nested heuristic for parameter tuning in Support Vector Machines (2014, methodology): “Support Vector Machines (SVM) [4, 9, 15, 46, 47] is a Supervised Classification technique rooted in Statistical Learning Theory [46, 47], whose success is based on the ability of building nonlinear classifiers.”
Sparsity in Optimal Randomized Classification Trees (2020, background): “The mainstream trend of using a greedy strategy in the onstruction of decision trees may lead to myopic decisions, hich, in turn, may affect the overall learning performance. he major advances in Mathematical Optimization ( Carrizosa & omero Morales, 2013; Olafsson, Li, & Wu, 2008; Silva, 2017 ) hav”
Global optimization issues in deep network regression: an overview (2018, methodology): “We refer to [41] for a survey on semi-supervised methods which are not covered in this paper, to [36,53] and references therein for complete surveys on mathematical optimization for SVM, and to [107] for a survey on solution approaches for the convex training problem of different SVMs.”
Clustering Categories in Support Vector Machines (2017, methodology): “First, we present the standard SVM formulation, [8, 10, 23, 24].”
Supersparse Linear Integer Models for Interpretable Classification (2013, background): “The review paper of Carrizosa and Romero Morales [15] mentions a way to extract easy-to-understand “if, then” rules from SVMs.”
On sparse ensemble methods: An application to short-term predictions of the evolution of COVID-19 (2021, methodology): “The consistency of the bootstrap for Support Vector Machines when the data can be assumed to be independent and identically distributed, has been shown in Christmann and Hable (2013) .”
On handling indicator constraints in mixed integer programming (2016, unknown): “The reader is referred to [9] for a recent reviewonMathematicalOptimization and SVMs.”
Optimization Models for Machine Learning: A Survey (2019, background): “where d is the degree of the polynomial, radial basis function K(xi, xj) = e − ‖xi−xj‖ 2 2 γ , and sigmoidal K(xi, xj) = tanh(φxi xj + c) [58, 120].”
Enhancing Interpretability in Factor Analysis by Means of Mathematical Optimization (2019, background): “(2008), or Information Visualization, Carrizosa et al. (2018a,b); von Landesberger et al. (2017). In this paper, we focus on latent variable models to uncover the hidden information enclosed in raw data, Anderson (2003); Bartholomew et al. (2011); Blei (2014).”
Visualizing data as objects by DC (difference of convex) optimization (2017, background): “Mathematical Optimization has contributed significantly to the development of this area during recent years, see [13, 30, 42] and the references therein.”