Evolutionary tuning of multiple SVM parameters

By Frauke Friedrichs, Christian Igel. The European Symposium on Artificial Neural Networks, 2005

548 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/b4eb71cce6b2bf1989a79eff679f413456a16157

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Optimizing the hyper-parameters for SVM by combining evolution strategies with a grid search (2006, methodology): “In some papers, the hyperparameters also were called multiple parameters which comprise of a regularization parameter [ 3 ] (soft margin parameter [2], penalty parameter in [4]) and kernel parameters.”
Trends and Applications in Constructive Approximation Evolutionary Optimization of Neural Systems: the Use of Strategy Adaptation (unknown): “We sketch applications of these methods to the design of different types of neural systems: topology optimization of multilayer NNs for face detection [49], weight optimization of recurrent networks for solving reinforcement learning tasks [21], and model selection for support vector machines [14].”
Meta-learning for evolutionary parameter optimization of classifiers (2012, methodology): “Friedrichs and Igel (2005) used this method to optimize multiple parameters of Support Vector Machines (SVM).”
Combining meta-learning and search techniques to select parameters for support vector machines (2012, background): “Another limitation of this solution is that the search process usually starts with random conﬁgurations uniformly sampled from the search space.”
A nested heuristic for parameter tuning in Support Vector Machines (2014, unknown): “This explains the variety of models, with different levels of generality, that can be found in the literature, and the need for a tuning method to be able to adapt to them.”
A trainable feature extractor for handwritten digit recognition (2007, unknown): “Some authors proposed other methods including evolutionary tuning [8] or gradient-based approaches [9][10] for tuning of SVM hyperparameters.”
SVM Parameter Optimization using Grid Search and Genetic Algorithm to Improve Classification Performance (2016, background): “Parameter optimization can be very time consuming if done manually especially when the learning algorithm has many parameters [9, 10].”
A Generalized Framework for Population Based Training (2019, background): “ed, have been widely applied to a variety of applications at scale. For hyperparameter tuning in machine learning models, the covariance matrix adaptation evolution strategy (CMA-ES) has been popular [5, 7]. Recently, large scale distributed evolution has proven effective in the neural architecture ”
A support vector machine–firefly algorithm-based model for global solar radiation prediction (2015, background): “In the past, although some researchers have applied various conventional optimization algorithms to select these parameters, the achieved results have not been so effective due to the complex nature of the parameters (Bao et al., 2013; Friedrichs and Igel, 2005; Lorena and De Carvalho, 2008).”
Evolutionary algorithms for hyperparameter tuning on neural networks models (2014, methodology): “In hyperparameter tuning problems, EA have been proved to perform better than grid search techniques based on accuracy-speed ratio (Martino et al. 2011; Friedrichs and Igel 2005; Huang and Wang 2006).”
Scalable Gaussian process-based transfer surrogates for hyperparameter optimization (2017, methodology): “Thesemethods are usually based on genetic algorithms (Friedrichs and Igel 2005a; de Souza et al. 2006), although some are deterministic (Keerthi et al. 2007).”
Selection of time instants and intervals with Support Vector Regression for multivariate functional data (2020, background): “The increase in the number of parameters makes grid search impractical and unreliable, hence calling for bespoke optimization-based approaches to tuning, Carrizosa et al. (2014), Chapelle et al. (2002) and Friedrichs and Igel (2005).”
A New Neural Dynamic Classification Algorithm (2017, methodology): “Friedrichs and Igel [26] improve SVM by using a Gaussian kernel function and optimizing its spread parameter employing a covariance matrix adaptation evolution strategy.”
Natural Evolution Strategies (2008, methodology): “It has been noticed by Glasmachers et al. (2010b) that in first order Taylor approximation the exponential xNES update coincides with the so-called rank-μ update of CMA-ES.”
Hyperparameter Search Space Pruning - A New Component for Sequential Model-Based Hyperparameter Optimization (2015, methodology): “Furthermore, there also exist strategies to optimize hyperparameters that are based on optimization techniques from arti cial intelligence such as tabu search [4], particle swarm optimization [12] and evolutionary algorithms [10] as well as gradient-based optimization techniques [6] designed for SVM”
An Experimental Study of the Combination of Meta-Learning with Particle Swarm Algorithms for SVM Parameter Selection (2012, methodology): “Different authors have deployed search and optimization techniques aiming to automatize this process and to avoid an exhaustive or a random exploration of parameters [14][4][7][8][9][12][13].”