On regularization algorithms in learning theory

By Frank Bauer, Sergei V. Pereverzyev, Lorenzo Rosasco. 2007

344 citations recorded by the source corpus.

Subfields: Applied Mathematics, Computational Theory and Mathematics, Mathematical Physics, Optimization & Training, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Mathematics

In Computational Theory and Mathematics

In Mathematical Physics

In Optimization & Training

In Statistical Learning Theory

Citations in the corpus

Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, methodology): “Later, Muandet et al. (2014b) provides a non-linear extension of the estimator in (3.22) by means of spectral filtering algorithms (Bauer et al. 2007, De Vito et al. 2006).”
Kernel Mean Shrinkage Estimators (2014, background): “Moreover, Proposition 12 reveals that the eﬀect of shrinkage is akin to spectral ﬁltering (Bauer et al., 2007)— which in our case corresponds to Tikhonov regularization—wherein S-KMSE ﬁlters out the high-frequency components of the spectral representation of the kernel mean.”
Density Estimation in Infinite Dimensional Exponential Families (2013, methodology): “(i) The proof follows the ideas in the proof of Theorem 10 in [3], which is a more general result dealing with the smoothness condition, f 0 ∈ R (Θ( C )) where Θ is operator monotone. self-adjoint operators Recall that Θ is operator monotone on [0 , b ] if for any pair of self-adjoint U , V with…”
Multi-output learning via spectral filtering (2012, methodology): “3.2 Beyond Tikhonov: regularization via spectral filtering In this section, we present the class of regularized kernel methods which we apply to multioutput learning, referring to the papers by Lo Gerfo et al. (2008), Bauer et al. (2007) for the scalar case.”
Less is More: Nyström Computational Regularization (2015, methodology): “principal component regression or truncated SVD) [30, 31], as well as regularized stochastic gradient methods [32].”
Gradient-Based Kernel Dimension Reduction for Regression (2014, background): “These facts have been proved in various contexts (e.g. Smale and Zhou, 2005, 2007, Caponnetto and De Vito, 2007, Bauer et al., 2007), so the proof is omitted.”
Generalization Properties of Learning with Random Features (2016, methodology): “This approach is classical in inverse problems [18] and has been proposed and analyzed in [25, 5] for machine learning.”
Simple and Effective Regularization Methods for Training on Noisily Labeled Data with Generalization Guarantee (2019, background): “It was shown to perform comparably to early-stopped gradient descent (Bauer et al., 2007; Gerfo et al., 2008; Raskutti et al., 2014; Wei et al., 2017).”
Elastic-net regularization in learning theory (2008, result): “which gives the same rate as in Theorem 2 of [43] and Corollary 17 of [44].”
Data-driven calibration of linear estimators with minimal penalties (2009, neutral): “Finally, convergence rates of some linear spectral estimators have recently been proved optimal [12, 8].”
Kernel ridge vs. principal component regression: Minimax bounds and the qualification of regularization operators (2017, methodology): “This has been observed previously by other researchers; others have even noted that Tikhonov regularization (KRR) saturates, while spectral cut-off regularization (KPCR) does not (Mathé, 2005; Bauer et al., 2007; Lo Gerfo et al., 2008).”
Vector Field Learning via Spectral Filtering (2010, methodology): “Each algorithm is characterized by specific constants that might change from one algorithm to the other and we refer to [22] for their computation.”
On Learning with Integral Operators (2010, methodology): “We note that the approach using concentration inequalities in a Hilbert space has already been proved useful for analyzing ROSASCO, BELKIN AND DE VITO supervised kernel algorithms, see De Vito et al. (2005b), Yao et al. (2007), Bauer et al. (2007) and Smale and Zhou (2007).”
Distributed Learning with Regularized Least Squares (2016, background): “(b) For any 0 δ 1, with confidence at least 1− δ, there holds ∥∥∥∥∥(LK + λI) −1/2 ( 1 |D| ∑ z∈D ξg(z)− E [ξg] )∥∥∥∥∥ K ≤ 2‖g‖∞ log(2/δ)√ |D| { κ√ |D|λ + √ N (λ) } .”
Optimal Rates for Regularization Operators in Learning Theory (2006, methodology): “In the context of learning theory a review of these techniques can be found in [10] and [1].”
Quantum machine learning: a classical perspective (2017, methodology): “A variety of regularization strategies have been proposed in the literature, each adopting a different perspective on the problem (see [11,21,22] for an introduction on the main ideas).”
Early stopping for non-parametric regression: An optimal data-dependent stopping rule (2011, methodology): “ Ste is used to estimate the risk via R HO(ft) = 1 n P i∈Ste yi −ft tr(xi) 2 , which deﬁnes the stopping rule Tb HO := argmin ˆ t ∈ N | R HO(ft+1 tr ) > R HO(f t tr) ˙ −1. (30) A line of past work [42, 7, 13, 11, 12, 37]) has analyzed stopping rules based on this type of holdout rule. For instanc”