Optimal regression rates for SVMs using Gaussian kernels
By Mona Eberts, Ingo Steinwart. 2013
87 citations recorded by the source corpus.
Support vector machines (SVMs) using Gaussian kernels are one of the standard and state-of-the-art learning algorithms. In this work, we establish new oracle inequalities for such SVMs when applied to either least squares or conditional quantile regression. With the help of these oracle inequalities we then derive learning rates that are (essentially) minmax optimal under standard smoothness assumptions on the target function. We further utilize the oracle inequalities to show that these learning rates can be adaptively achieved by a simple data-dependent parameter selection method that splits the data set into a training and a validation set.
Subfields: Artificial Intelligence, Statistics and Probability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Kenji Fukumizu
- Yu Nishiyama
- Motonobu Kanagawa
- Arthur Gretton
- Bernhard Schölkopf
- Bharath K. Sriperumbudur
- Marie Davidian
- Krikamol Muandet
In Statistics and Probability
- Eric B. Laber
- Marie Davidian
- Kenji Fukumizu
- Anastasios A. Tsiatis
- Arthur Gretton
- Yichi Zhang
- Nathan Kallus
- Motonobu Kanagawa
Citations in the corpus
- Filtering with State-Observation Examples via Kernel Monte Carlo Filter (2013, methodology): “n in the literature on kernel mean embeddings (e.g., Theorem 5 of Fukumizu et al. (2013)), we may relax this assumption by using approximate arguments in learning theory (e.g., Theorem 2.2 and 2.3 of Eberts and Steinwart (2013)). This analysis remains a topic for future research. 5 To show this, it ”
- Interpretable Dynamic Treatment Regimes (2016, methodology): “ t . 16 The minimax convergence rate for a nonparametric regression estimator of an r T-times continuously dierentiable function is Ofn (2r T)=(2r T+d T)g(Stone, 1982). By extending the technique in Eberts and Steinwart (2013), we show in the Supplemental Material that the estimated Q-function Qb T ”
- Monte Carlo Filtering Using Kernel Embedding of Distributions (2014, background): “Note that assumption θ ∈ HX ⊗ HX is a technical one to simplify the assertion and the involved proof, and can be weakened by using approximation arguments of statistical learning theory, e.g., (Eberts and Steinwart 2013).”
- Recovering Distributions from Gaussian RKHS Embeddings (2014, background): “A recent study on learning theory has yielded bounds for errors when approximating a Besov function with certain RKHS functions and for their associated RKHS norms (Eberts and Steinwart, 2013, Theorem 2.2., Theorem 2.3).”