Regularization in kernel learning

By Shahar Mendelson, Joe Neeman. 2010

158 citations recorded by the source corpus.

Supported in part by Australian Research Council Discovery Grant DP0559465 and by Israel Science Foundation Grant 666/06.

Subfields: Artificial Intelligence, Mathematical Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Mathematical Physics

Citations in the corpus

On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions (2015, methodology): “2 op kΣ−1/2hk L2(dρ) 62λ 1/2−r. When r= 1/2, we get a result in the RKHS norm, but with no decay to zero; the RKHS norm k·kF would allow acontrol in L∞-norm, but as noticed bySteinwart et al. (2009); Mendelson and Neeman (2010), such a control may be obtained in practice with rmuch smaller. For exam”
Learning Theory for Distribution Regression (2014, background): “ciated to H. Indeed, there exist several results along these lines with KRR for the case of real-valued outputs: see for example (Sun and Wu, 2009a, Theorem 1.1), (Sun and Wu, 2009b, Corollary 3.2), (Mendelson and Neeman, 2010, Theorem 3.7 with Assumption 3.2). The question of optimal rates has also”
Kernel Mean Shrinkage Estimators (2014, methodology): “Tk induced by the kernel kand the approximation properties of H. This link has been extensively used for the analysis of some state-of-the-art learning algorithms, see, e.g., Cucker and Smale (2002); Mendelson and Neeman (2010) and references therein, and will prove helpful in understanding several ”
Efficient Policy Learning (2017, methodology): “Here, one option is to use non-parametric estimators for which we can precisely spell out when they satisfy Assumption 2, such as sieve-based methods (Chen, 2007; Negahban, Ravikumar, Wainwright, and Yu, 2012) or kernel regression (Caponnetto and De Vito, 2007; Mendelson and Neeman, 2010).”
Quasi-oracle estimation of heterogeneous treatment effects (2017, methodology): “Mendelson and Neeman (2010) then conclude their argument by noting that, for any 1 ≤ c ≤ C (see their Corollary 2.7 for details 6 ), and that by optimizing this bound (up to logarithmic factors) with C = ∞ and a speciﬁc choice of ρ n ( · ), and by setting c to be c n = n α/ ( p +(1 − 2 α )) , we…”
Estimating conditional quantiles with the help of the pinball loss (2011, methodology): “Recall that (5) was first used in [6] to establish an oracle inequality for SVMs using the hinge loss, while [7, 18, 25] consider (5) for SVMs using the least-squares loss.”
ℓ1-regularized linear regression: persistence and oracle inequalities (2011, methodology): “A proof of this fact is given in [29] when Fr is the unit ball of a reproducing kernel Hilbert space, but the same argument works in our case and so we omit it.”
On the Equivalence between Quadrature Rules and Random Features (2015, methodology): “We have: When r = 1 / 2, we get a result in the RKHS norm, but with no decay to zero; the RKHS norm · k F would allow a control in L ∞ -norm, but as noticed by Steinwart et al. (2009); Mendelson and Neeman (2010), such a control may be obtained in practice with r much smaller.”
Simultaneous Model Selection and Optimization through Parameter-free Stochastic Learning (2014, methodology): “It has been first proposed for classification by [5], and for regression by [17].”
Optimal Rates for Regularized Least Squares Regression (2009, background): “Modulo some extra logarithmic factors in the regularization term, this has already been observed by [13].”
Optimal regression rates for SVMs using Gaussian kernels (2013, methodology): “Here, we only note that besides a very few articles, namely [5, 22, 32], the obtained learning rates are typically not optimal in a minmax sense.”
Distributed Learning with Regularized Least Squares (2016, background): “…functions spaces L 2 ρ X and H K , and to take the projection onto [ M, M ], although (14) is more general than the uniform boundedness assumption of the eigenfunctions and holds when H K is the Sobolev space and ρ X is the uniform distribution (Steinwart et al., 2009; Mendelson and Neeman, 2010).”
Mercer’s Theorem on General Domains: On the Interaction between Measures, Kernels, and RKHSs (2012, background): “It has been recently discovered in [20, 37] that the statistical analysis of SVMs can be tightened for RKHSs that allow a refined control of the ‖ ·‖∞-norm of their functions in the sense of (39).”
Offline Multi-Action Policy Learning: Generalization and Optimization (2018, background): “For more, see Mendelson et al. (2010).”
Less is More: Nyström Computational Regularization (2015, background): “Techniques known to achieve optimal error rates include Tikhonov regularization [26, 28, 29], iterative regularization by early stopping [30, 31], spectral cut-off regularization (a.”