Learning with Kernels: support vector machines, regularization, optimization, and beyond

By Bernhard Schölkopf, Alex Smola. 2001

5,369 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical Learning Theory

Citations in the corpus

Discriminative Embeddings of Latent Variable Models for Structured Data (2016, positive): “The success of kernel methods on structured data relies crucially on the design of kernel functions — positive semidefinite similarity measures between pairs of data points (Schölkopf & Smola, 2002).”
Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning (2017, background): “Supervised learning algorithms like Support Vector Machines (SVMs) [60], Random forests [7], Gaussian processes (GPs) [55] and, of course, deep neural networks [25] can be depicted by this general framework.”
Taking Advantage of Multitask Learning for Fair Classification (2018, neutral): “[33 34] and references therein.”
Kernel-based Conditional Independence Test and Application in Causal Discovery (2011, methodology): “We use kernel ridge regression to estimate the regression function h ∗ f ( Z ) in (4), and one can easily see that ˆ h ∗ f , where ε is a small positive regularization parameter (Sch¨olkopf and Smola, 2002).”
Adaptivity and Computation-Statistics Tradeoffs for Kernel and Distance based High Dimensional Two Sample Testing (2015, background): “1 The Median Heuristic The median heuristic chooses the bandwidth for the Gaussian kernel as the median pairwise distance between all pairs of points (see Schölkopf and Smola [2002]).”
Learning Theory for Distribution Regression (2014, background): “A more theoretically grounded approach to learning on distributions has been to define positive definite kernels (Schölkopf and Smola, 2002) on the basis of statistical divergence measures on distributions, or by metrics on non-negative numbers; these can then be used in kernel algorithms.”
On the Decreasing Power of Kernel and Distance Based Nonparametric Hypothesis Tests in High Dimensions (2014, methodology): “One of the most common ways in the literature to choose this bandwidth is using the median heuristic, see (Schölkopf and Smola 2002), according to which γ is chosen to be the median of all pairwise distances.”
Identifying Finite Mixtures of Nonparametric Product Distributions and Causal Inference of Confounders (2013, methodology): “Under the assumption of linear relationships between variables and non-Gaussian distributions, confounders may be identified using Independent Component Analysis (Shimizu et al., 2009). Other results for the linear case are presented in Silva et al. (2006) and for the non-linear case with additive n”
Sharp analysis of low-rank kernel matrix approximations (2012, positive): “First, using the representer theorem, the unique solution f may be found as f = ∑n i=1 αiφ(xi) (see, e.g., Wahba, 1990; Schölkopf and Smola, 2002; Shawe-Taylor and Cristianini, 2004).”
Kernel Bayes' Rule (2010, background): “Kernel methods have long provided powerful tools for generalizing linear statistical approaches to nonlinear settings, through an embedding of the sample to a high dimensional feature space, namely a reproducing kernel Hilbert space (RKHS) [16].”
Universality, Characteristic Kernels and RKHS Embedding of Measures (2010, methodology): “They are broadly established as an easy way to construct nonlinear algorithms from linear ones, by embedding data points into higher dimensional reproducing kernel Hilbert spaces (RKHSs) (Sch¨olkopf and Smola, 2002; Shawe-Taylor and Cristianini, 2004).”
On the empirical estimation of integral probability metrics (2012, methodology): “In the following, we present another interpretation for the kernel distance by relating it to the Parzen window classifier [37, 38] (also called the kernel classification rule [14]).”
Video Representation Learning Using Discriminative Pooling (2018, background): “Assuming X = X+∪X−, by the Representer theorem [43], it is wellknown that for a kernel K : X × X → R+, the decision function f for the SVM problem P1 will be of the form:”