SPARSITY IN MULTIPLE KERNEL LEARNING

By Vladimir Koltchinskii, Ming Yuan. 2010

201 citations recorded by the source corpus.

The problem of multiple kernel learning based on penalized empirical risk minimization is discussed. The complexity penalty is determined jointly by the empirical L2 norms and the reproducing kernel Hilbert space (RKHS) norms induced by the kernels with a data-driven choice of regularization parameters. The main focus is on the case when the total number of kernels is large, but only a relatively small number of them is needed to represent the target function, so that the problem is sparse. The goal is to establish oracle inequalities for the excess risk of the resulting prediction rule showing that the method is adaptive both to the unknown design distribution and to the sparsity of the problem.

Subfields: Civil and Structural Engineering, Computational Mechanics, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Civil and Structural Engineering

In Computational Mechanics

In Statistics and Probability

Citations in the corpus

Generalized Sparse Additive Models (2019, background): “However, previous proposals consists of gaps around eﬃcient computation [Koltchinskii and Yuan, 2010, Raskutti et al., 2012, Yuan and Zhou, 2015] and/or optimal statistical convergence properties [Ravikumar et al., 2009, Lou et al., 2016, Petersen et al., 2016, Sadhanala and Tibshirani, 2018].”
Additive models with trend filtering (2017, methodology): “Some nice contributions are given in Lin and Zhang [2006], Ravikumar et al. [2009], Meier et al. [2009], all primarily focused on fitting splines for component functions and achieving sparsity through a group lasso type penalty. In other even more recent and interesting work sparse additive models, ”
Semiparametric regression models with additive nonparametric components and high dimensional parametric components (2012, background): “Meier et al. (2009) and Koltchinskii and Yuan (2010), despite the difference in the forms of their penalties, both proposed penalties combining the empirical L2-norm and the usual roughness norm to enforce both sparsity and smoothness.”
Learning Sparse Additive Models with Interactions in High Dimensions (2016, background): “[12, 28] propose a convex program for estimating f (in the RKHS setting) that achieves the minimax optimal error rates.”
A unified framework for high-dimensional analysis of $M$-estimators with decomposable regularizers (2009, background): “Similarly, there are other nonparametric settings in which a more delicate choice of the regularization parameter is required [34, 57].”
Local Inference in Additive Models with Decorrelated Local Linear Estimator (2019, methodology): “Here we introduce one version of the theoretical restricted eigenvalue, which was used in [23, 37].”
Improved Bounds for the Nyström Method With Application to Kernel Classification (2011, unknown): “To this end, we first introduce the definition of the power law distribution of eigenvalues [28], [29].”
Faithful Variable Screening for High-Dimensional Convex Regression (2014, background): “Variable selection over kernel classes is studied by Koltchinskii and Yuan [15].”
Scalable Matrix-valued Kernel Learning for High-dimensional Nonlinear Multivariate Regression and Granger Causality (2012, methodology): “As such our framework may be viewed as providing generalizations of scalar multiple kernel learning [20, 26, 21, 29] and associated structured sparsity algorithms [8].”
Finding and Leveraging Structure in Learning Problems (2012, methodology): “Recently, several authors have shown how sparse additive models for regression can be efficiently estimated even when p > n [115, 139, 156, 157].”
Group Lasso for high dimensional sparse quantile regression models (2011, methodology): “The application of the group Lasso (and its variants) to the estimation of non-parametric additive models has recently gained a lot of attention (Ravikumar et al., 2009; Meier et al., 2009; Huang et al., 2010; Koltchinskii and Yuan, 2010; Raskutti et al., 2010).”