Empirical Bayes selection of wavelet thresholds

By Iain M. Johnstone, Bernard Walter Silverman. 2005

375 citations recorded by the source corpus.

This paper explores a class of empirical Bayes methods for level-dependent threshold selection in wavelet shrinkage. The prior considered for each wavelet coefficient is a mixture of an atom of probability at zero and a heavy-tailed density. The mixing weight, or sparsity parameter, for each level of the transform is chosen by marginal maximum likelihood. If estimation is carried out using the posterior median, this is a random thresholding procedure; the estimation can also be carried out using other thresholding rules with the same threshold. Details of the calculations needed for implementing the procedure are included. In practice, the estimates are quick to compute and there is software available. Simulations on the standard model functions show excellent performance, and applications to data drawn from various fields of application are used to explore the practical performance of the approach. By using a general result on the risk of the corresponding marginal maximum likelihood approach for a single sequence, overall bounds on the risk of the method are found subject to membership of the unknown function in one of a wide range of Besov classes, covering also the case of f of bounded variation. The rates obtained are optimal for any value of the parameter p in (0, ∞], simultaneously for a wide range of loss functions, each dominating the Lq norm of the σth derivative, with σ ≥ 0 and 0 < q ≤ 2. Attention is paid to the distinction between sampling the unknown function within white noise and sampling at discrete points, and between placing constraints on the function itself and on the discrete wavelet transform of its sequence of values at the observation points. Results for all relevant combinations of these scenarios are obtained. In some cases a key feature of the theory is a particular boundary-corrected wavelet basis, details of which are discussed. Overall, the approach described seems so far unique in combining the properties of fast computation, good theoretical properties and good performance in simulations and in practice. A key feature appears to be that the estimate of sparsity adapts to three different zones of estimation, first where the signal is not sparse enough for thresholding to be of benefit, second where an appropriately chosen threshold results in substantially improved estimation, and third where the signal is so sparse that the zero estimate gives the optimum accuracy rate.

Subfields: Applied Mathematics, Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/e113416d1b0b2473dbc55f443078584d97926ce3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Mathematics

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector (2013, neutral): “Examples include variable selection in regression (Fan and Lv 2010), covariance matrix estimation (Cai et al. 2010; Cai and Zhou 2012; Lam and Fan 2009), large-scale multiple testing (Bogdan et al. 2011; Cai and Jin 2010), and function estimation (Cai 2012; Johnstone and Silverman 2005).”
Mixtures of g Priors for Bayesian Variable Selection (2008, background): “For the large-p, small-n setting, independent priors on regression coefficients have become popular for inducing shrinkage (Wolfe, Godsill, and Ng 2004; Johnstone and Silverman 2005).”
Rao–Blackwellization for Bayesian Variable Selection and Model Averaging in Linear and Binary Regression: A Novel Data Augmentation Approach (2011, background): “Johnstone and Silverman (2004, 2005) also demonstrate the superior performance of mixtures of a point mass and heavy-tailed prior distributions compared to mixtures of a point mass and Gaussian prior distributions.”
Generalized fiducial inference for wavelet regression (2009, methodology): “The proposed methods for estimating and constructing confidence intervals for g are developed from the wavelet thresholding method of Lee (2002) and the generalized fiducial approach of Hannig (2009).”
Bayes procedures for adaptive inference in inverse problems for the white noise model (2012, neutral): “Rigorous results about the asymptotic behavior of empirical Bayes selectors of hyper-parameters in inﬁnite-dimensional problems only exist for a limited number of special problems, see e.g. [ 4,19,20,35,40].”
Adapting to Unknown Smoothness by Aggregation of Thresholded Wavelet Estimators (2006, methodology): “In fact, the multi-thresholding estimator has similar minimax performances than the empirical Bayes wavelet methods (see [55] and [32]) and several term-by-term wavelet thresholded estimators defined with a random threshold (see [33] and [7]).”
Testing in mixed-effects FANOVA models (2006, result): “, 2 − 1, or H0 : ∞j=0 ∑2j−1 k=0 μ 2 jk = 0, where we drop the scaling coefficient μ−10 since due to the identifiability condition ∫ μ(t)dt = 0 (see (4)) it is zero under both hypotheses. The proposed test will be similar in spirit to those of Spokoiny (1996) and Abramovich et al.”
BAYESIAN HYPER‐LASSOS WITH NON‐CONVEX PENALIZATION (2011, background): “This form is also given as (13) in Johnstone & Silverman (2005).”
A Probabilistic Model for Dirty Multi-task Feature Selection (2015, background): “Johnstone & Silverman (2005) provide the analytic solution when the variance of the Gaussian is one.”
The Role of Pseudo Data for Robust Smoothing with Application to Wavelet Regression (2007, methodology): “† EBayes: the empirical Bayes thresholding method of Johnstone and Silverman (2005) ,”
Real nonparametric regression using complex wavelets (2004, methodology): “ Johnstone and Silverman (2002a) we employ an automatic empirical Bayes approach which”
Multiscale methods for data on graphs and irregular multidimensional situations (2009, methodology): “If the ci are not all the same, then we apply the ‘parametric dependence’ approach within each artificial level as described in Johnstone and Silverman (2005a).”