Testing composite hypotheses, Hermite polynomials and optimal estimation of a nonsmooth functional

By Tommaso Cai, Mark G. Low. 2011

91 citations recorded by the source corpus.

A general lower bound is developed for the minimax risk when estimating an arbitrary functional. The bound is based on testing two composite hypotheses and is shown to be effective in estimating the nonsmooth functional ${\frac{1}{n}}\sum|\theta_i|$ from an observation $Y\sim N(\theta,I_n)$. This problem exhibits some features that are significantly different from those that occur in estimating conventional smooth functionals. This is a setting where standard techniques fail to yield sharp results. A sharp minimax lower bound is established by applying the general lower bound technique based on testing two composite hypotheses. A key step is the construction of two special priors and bounding the chi-square distance between two normal mixtures. An estimator is constructed using approximation theory and Hermite polynomials and is shown to be asymptotically sharp minimax when the means are bounded by a given value $M$. It is shown that the minimax risk equals $\beta_*^2M^2({\frac{\log\log n}{\log n}})^2$ asymptotically, where $\beta_*$ is the Bernstein constant. The general techniques and results developed in the present paper can also be used to solve other related problems.

Subfields: Artificial Intelligence, Control and Systems Engineering, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Statistics and Probability

Citations in the corpus

Chebyshev polynomials, moment matching, and optimal estimation of the unseen (2015, methodology): “The technique of polynomial approximation has been previously used for estimating non-smooth functions (Lq-norms) in Gaussian models [19, 22, 4] and more recently for estimating information quantities (entropy and power sums) on large discrete alphabets [43, 20].”
Minimax Estimation of Functionals of Discrete Distributions (2014, methodology): “It is shown in Cai and Low [55, Lemma 2] that all of the coefficients of Chebyshev polynomial T2m(x),m ∈ Z+ are upper bounded by 23m.”
Minimax Rates of Entropy Estimation on Large Alphabets via Best Polynomial Approximation (2014, methodology): “Combining (52)–(53), we obtain A Non-asymptotic risk bounds for the plug-in estimator Recall the worst-case quadratic risk of the plug-in estimator R plug-in ( k, n ) deﬁned in (4).”
Minimax Rate-optimal Estimation of KL Divergence between Discrete Distributions (2016, unknown): “, the scaled 1 norm 1 n ∑n i=1 |θi| in the Gaussian model [11], the Shannon entropy ∑S i=1 −pi ln pi [1], [12]–[14], the mutual information [1], the power sum function ∑S i=1 p α i [1], the Rényi entropy ln ∑S i=1 p α i 1−α [15] and the L1 distance ∑S i=1 |pi − qi| [16] in Multinomial and Poisson mo”
Efficient multivariate entropy estimation via $k$-nearest neighbour distances (2016, background): “Very recently, in independent work and under regularity conditions, Delattre and Fournier (2017) derived the asymptotic normality of the estimator when k = 1, confirming the suboptimal asymptotic variance in this case. Previous works on the general k case include Singh et al. (2003), where heuristic”
A Unified Maximum Likelihood Approach for Estimating Symmetric Properties of Discrete Distributions (2017, methodology): “To bound the largest deviation, we use the fact from Lemma 2 in (Cai et al., 2011) that the largest coefficient of the best degree-L polynomial approximation of |x| in [−1, 1] has all coefficients at most 23L. Similar argument as with entropy yields that after appropriate normalization, the largest…”