Learning Theory Estimates via Integral Operators and Their Approximations

By Steve Smale, Ding-Xuan Zhou. 2007

687 citations recorded by the source corpus.

Subfields: Applied Mathematics, Mathematical Physics, Statistics and Probability, Probabilistic & Bayesian Methods, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Mathematics

In Mathematical Physics

In Statistics and Probability

In Probabilistic & Bayesian Methods

In Statistical Learning Theory

Citations in the corpus

Convergence guarantees for kernel-based quadrature rules in misspecified settings (2016, neutral): “ Therefore the assumption f∈ Hθ is equivalent to f∈ R(Tθ2 ). For each n, deﬁne a constant λn = n−2b+2c−1. Deﬁne a function f λn ∈ H by fλn := (T+λnI)−1Tf, where Idenotes the identity. From Lemma 3 of [22] and f∈ R(Tθ2 ), this function satisﬁes kf−fλnkL 2(Q) ≤ λ θ 2 nkT −θ 2fkL 2(Q), (18) kfλnkH ≤ λ ”
Density Estimation in Infinite Dimensional Exponential Families (2013, methodology): “While the range space assumption has been made in various earlier works (e.g., [8, 32, 19] in the context of non-parametric least square regression), in the following, we investigate the implicit smoothness assumptions that it makes on f 0 in our context.”
Kernel Bayes' Rule (2010, methodology): “The proof idea for the consistency rates of the KBR estimators is similar to [6, 30], in which the basic techniques are taken from the general theory of regularization [8].”
Optimal Rates for the Regularized Least-Squares Algorithm (2007, methodology): “Although the previous investigations in [8], [26], [9], [27] showed that operator and spectral methods are valuable tools for the performance analysis of kernelbased algorithms such as RLS, all these results failed to compare with similar results recently obtained using entropy methods (see [10], [2”
Kernel methods in system identification, machine learning and function estimation: A survey (2014, background): “Smale and Zhou (2007) and Wu, Ying, and Zhou (2006), also in the context of NARX identification (De Nicolao & Trecate, 1999).”
Elastic-net regularization in learning theory (2008, result): “which gives the same rate as in Theorem 2 of [43] and Corollary 17 of [44].”
Towards Understanding the Spectral Bias of Deep Learning (2019, methodology): “The theory of integral operator with respect to kernel function has been well studied in literature [Smale and Zhou, 2007; Rosasco et al., 2010] thus we only give a brief introduction here.”
Non-parametric Stochastic Approximation with Large Step sizes (2014, methodology): “data in Rd, d > 1, and so-called Mercer kernels (continuous kernels over a compact set) that lead to dense subspaces of the space of square-integrable functions and non parametric estimation [4], has been widely studied in the last decade starting with the works of Smale and Cucker [5, 6] and being ”
Beyond Least-Squares: Fast Rates for Regularized Empirical Risk Minimization through Self-Concordance (2019, positive): “Our main goal in this paper is to bridge the gap between Lipschitz-continuous and quadratic losses by improving on slow rates for general classes of losses beyond least-squares.”
On Early Stopping in Gradient Descent Learning (2007, methodology): “Here we recall some basic deﬁnitions on RKHSs and refer to Appendix B for an exposition with more details.”
Exact minimax risk for linear least squares, and the lower tail of sample covariance matrices (2019, background): “Another type of non-uniform guarantees over linear classes is achieved by Ridge regression [31, 70] in the context of finite-dimensional or nonparametric reproducing kernel Hilbert spaces [18, 19, 20, 14, 62, 65, 5, 33], where the bounds do not depend explicitly on the dimension d, but rather on spe”
Optimal Rates for Regularization Operators in Learning Theory (2006, background): “This type of condition has been introduced in the statistical learning literature in [7, 18, 3, 17, 4].”
Spectral Methods for Regularization in Learning Theory (2006, methodology): “The price we pay for our generality is that for the two algorithms already studied (see Smale and Zhou (2005), Caponnetto and De Vito (2005) for Tikhonov regularization and Yao et al. (2005) for gradient descent learning) the bounds we find do not match the best available bounds.”
A new kernel-based approach for linear system identification (2010, positive): “This result can be derived by extending the error analysis reported in [39].”
Gradient-Based Kernel Dimension Reduction for Regression (2014, background): “These facts have been proved in various contexts (e.g. Smale and Zhou, 2005, 2007, Caponnetto and De Vito, 2007, Bauer et al., 2007), so the proof is omitted.”