Introduction to Nonparametric Estimation

By Alexandre B. Tsybakov. 2008

3,467 citations recorded by the source corpus.

Subfields: Statistics and Probability, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

In Statistical Learning Theory

Citations in the corpus

When do neural networks outperform kernel methods? (2020, methodology): “[Tsy08] Alexandre B Tsybakov, Introduction to nonparametric estimation, Springer Science & Business Media, 2008.”
Distributed Simulation and Distributed Inference (2018, background): “With more than a century of history (see the books [Tsy09, DL01]), distribution learning has recently seen a surge of interest in the computer science community as well, with a focus on discrete domains (see e.g. [ Dia16] for a survey of these recent developments).”
Learning Anonymized Representations with Adversarial Neural Networks (2018, methodology): “The proof of this proposition follows by applying Lemma 2.10 in [Tsybakov, 2008] from which we can bound from below the misclassiﬁcation probability and will be omitted.”
Geometric Lower Bounds for Distributed Parameter Estimation Under Communication Constraints (2018, neutral): “By a standard testing argument Tsybakov (2008), we have inf ΠBB inf θ̂ sup θ∈Θ Eθ‖θ̂(Y )− θ‖22 ≥ dδ2 10 inf ΠBB inf Û P ( dHam(Û , U) ≥ d 5 ) where dHam(x, y) , ∑d i=1 1(xi 6= yi) denotes the Hamming distance.”
Optimal Schemes for Discrete Distribution Estimation Under Locally Differential Privacy (2017, methodology): “In the classical (non-private) minimax estimation problem, one standard approach to the proof of lower bounds on the minimax risk of estimation is Assouad’s method [18] (see also [19]).”
Extremal Mechanisms for Local Differential Privacy (2014, background): “In the hypothesis testing setup, the utility is a function of the privatization mechanism and two priors defined on the input alphabet.”
Double/De-Biased Machine Learning of Global and Local Parameters Using Regularized Riesz Representers (2018, background): “ √ n|Erα(X)rγ(X)| ≤ √ nkrαkP,2krγkP,2 being small follows from the fact that many rich functional classes admit sparse linear approximations with respect to conventional dictionaries b. For instance, Tsybakov (2012) and Belloni et al. (2014b) give examples of Sobolev and rearranged Sobolev balls, re”
Minimax Estimation of Kernel Mean Embeddings (2016, methodology): “…by the kernel density estimator p̂n (which is nothing but the empirical estimator of µKh(P )) as it is known (based on a straightforward generalization of Tsybakov, 2008, Proposition 1.4 for multiple dimensions) that E ∥ ∥p̂n − E[p̂n] ∥ ∥2 L2(Rd) ≤ ‖K‖2 L2(Rd) nhd , where we assume K ∈ L2(Rd).”
On the Decreasing Power of Kernel and Distance Based Nonparametric Hypothesis Tests in High Dimensions (2014, methodology): “Our proposal is guided by the fact that KL-divergence (and MI) is a fundamental information-theoretic quantity that is well-known to determine the hardness of hypothesis testing problems, for example via lower bounds using variants of Fano’s inequality, see (Tsybakov 2010).”
Density Estimation in Infinite Dimensional Exponential Families (2013, methodology): “By the minimax theory [38, Chapter 2], it is well known that for any α > β ≥ 0 , inf where the inﬁmum is taken over all possible estimators and ≍ means “proportional up to constants.””
Fast Distribution To Real Regression (2013, methodology): “l input distributions in a Holder class and a smooth class of mappings has been previously studied in [16]. There, an estimator{the Kernel-Kernel estimator{analogous to the Nadaraya-Watson estimator [20] for functional distribution inputs was shown to have a polynomial rate of convergence. This rate”
Generative and Latent Mean Map Kernels (2010, background): “An advantage of using kernel density estimation is that it is known to be consistent [27].”
Variance-based Regularization with Convex Objectives (2016, methodology): “We have by standard bounds on the KL-divergence [26, Lemma 2.7] that , so that For n odd, the ﬁnal probability is 0, while for n even, we have where the inequality uses that 2 ≤ 4 n √ πn by Stirling’s approximation.”
Minimax Optimal Procedures for Locally Private Estimation (2016, methodology): “…f ∈ F β [ C ] and that quality is measured in terms of the squared error (squared L 2 [0 , 1]-norm): The well-known (Yu 1997; Yang and Barron 1999; Tsybakov 2009) (nonprivate) minimax mean squared error scales as The goal of this section is to understand how this minimax rate changes when we add…”
Divide and conquer kernel ridge regression: a distributed algorithm with minimax optimal rates (2013, background): “Then the mean-squared error is bounded as The upper bound (19) is unimprovable up to constant factors, as shown by known minimax bounds on estimation error in Sobolev spaces (Stone, 1982; Tsybakov, 2009); see also Theorem 2(b) of Raskutti et al. (2012).”
Local Privacy and Minimax Bounds: Sharp Rates for Probability Estimation (2013, background): “(1) of Yang and Barron [29] or Chapter 2 of Cover and Thomas [8].”
From optimal transport to generative modeling: the VEGAN cookbook (2017, neutral): “Furthermore, D JS ( P, Q ) ≤ D TV ( P, Q ) [14, Theorem 3] and D TV ( P, Q ) ≤ D KL ( P, Q ) [15, Eq.”