Choosing Multiple Parameters for Support Vector Machines

By Olivier Chapelle, Vladimir Naumovich Vapnik, Olivier Bousquet, Sayan Mukherjee. 2002

2,403 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Optimization & Training, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Optimization & Training

In Statistical Learning Theory

Citations in the corpus

Gradient-based Hyperparameter Optimization through Reversible Learning (2015, background): “Chapelle et al. (2002) introduced a differentiable bound on the SVM loss in order to be able to compute derivatives with respect to hundreds of hyperparameters, including weighting parameters for each input dimension in the kernel.”
Hyperparameter optimization with approximate gradient (2016, methodology): “…setting the regularization parameter in the context of neural networks (Larsen et al., 1996; 1998; Bengio, 2000), has also been applied to the problem of selecting kernel parameters (Chapelle et al., 2002; Seeger, 2008) or multiple regularization parameters in log-linear models (Foo et al., 2008).”
Benchmark and Survey of Automated Machine Learning Frameworks (2019, neutral): “For limited settings, e.g., tuning C and γ of a support-vector machine (SVM) (Momma & Bennett, 2002; Chapelle et al., 2002; Chen et al., 2004), it was proven that guided search strategies yield better results than grid search in less time.”
Efficient multiple hyperparameter learning for log-linear models (2007, methodology): “In SVMs, approaches based on optimizing generalization bounds [4], such as the radius/margin-bound [15] or maximal discrepancy criterion [2] have been proposed.”
Learning using privileged information: SV M+ and weighted SVM (2013, methodology): “The problem (5.4) is thus replaced with c (cid:63) = arg min This idea follows the method of (Chapelle et al., 2002) who suggested to tune L2-SVM parameters by minimizing certain estimates of the generalization error using a gradient descent algorithm.”
Hyperparameter Learning for Graph Based Semi-supervised Learning Algorithms (2006, methodology): “This idea is similar to [5] which learns multiple parameters for SVM.”
Classification of hyperspectral remote sensing images with support vector machines (2004, methodology): “Recently, two interesting automatic techniques have been developed to deal with this issue [37], [38].”
An Introduction to Boosting and Leveraging (2002, methodology): “Additionally, it would be interesting to develop bounds or efficient methods to compute the leave-one-out error as done for SVMs in [36].”
Bayesian Nonparametric Kernel-Learning (2015, methodology): “This is often done with cross-validation or with methods like [12, 6].”
Machine Learning and Radiology (2012, background): “A multiple kernel learning approach provides a feasible way to solve real applications which involve multiple, heterogeneous data sources (Chapelle et al., 2002; Lanckriet et al., 2004; Sonnenburg et al., 2006).”
Support vector machine in structural reliability analysis: A review (2023, positive): “Several algorithms based on different optimization techniques were developed to search the SVM hyperparameters e.g., gradient descent algorithm [32], coordinate descent method [33], grid search [34], five-fold cross-validation method [24], real-value genetic algorithm [35], particle swarm…”