Optimization Models for Machine Learning: A Survey

By Claudio Gambella, Bissan Ghaddar, Joe Naoum-Sawaya. 2019

261 citations recorded by the source corpus.

Abstract This paper surveys the machine learning literature and presents in an optimization framework several commonly used machine learning approaches. Particularly, mathematical optimization models are presented for regression, classification, clustering, deep learning, and adversarial learning, as well as new emerging applications in machine teaching, empirical model learning, and Bayesian network structure learning. Such models can benefit from the advancement of numerical optimization techniques which have already played a distinctive role in several machine learning settings. The strengths and the shortcomings of these models are discussed and potential research directions and open problems are highlighted.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Mathematical optimization in classification and regression trees (2021, background): “See Bot-tou et al. (2018), Gambella et al. (2020), Liberti (2020) for surveys reviewing the use of Mathematical Optimization in Machine Learning, and Carrizosa and Romero Morales (2013), Duarte Silva (2017), Palagi (2019), and Piccialli and Sciandrone (2018) for surveys focusing on specific…”
Puma optimizer (PO): a novel metaheuristic optimization algorithm and its application in machine learning (2024, unknown): “Nowa-days, meta-heuristic algorithms are applied for optimization operations in improving various machine learning and data mining models [72], which increase the performance of these models.”
Understanding of Machine Learning with Deep Learning: Architectures, Workflow, Applications and Future Directions (2023, background): “It has simpliﬁed life and become an indispensable instrument in several industries, such as agriculture [2], banking [3], optimisation [4], robotics [5], structural health monitoring [6], and so on.”
On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice (2020, unknown): “hitectures determined by dierent hyper-parameter congurations, the domain of the ML model 11 function fis restricted to a set of functions F. Thus, the optimal predictive model f can be obtained by [34]: f = argmin f2F 1 n Xn i=1 L(f(x i);y i) (7) where nis the number of training data points, x i is”