On the optimal estimation of probability measures in weak and strong topologies

By Bharath K. Sriperumbudur. 2013

69 citations recorded by the source corpus.

Given random samples drawn i.i.d. from a probability measure $\mathbb{P}$ (defined on say, $\mathbb{R}^d$), it is well-known that the empirical estimator is an optimal estimator of $\mathbb{P}$ in weak topology but not even a consistent estimator of its density (if it exists) in the strong topology (induced by the total variation distance). On the other hand, various popular density estimators such as kernel and wavelet density estimators are optimal in the strong topology in the sense of achieving the minimax rate over all estimators for a Sobolev ball of densities. Recently, it has been shown in a series of papers by Gin\'{e} and Nickl that these density estimators on $\mathbb{R}$ that are optimal in strong topology are also optimal in $\|\cdot\|_{\mathcal{F}}$ for certain choices of $\mathcal{F}$ such that $\|\cdot\|_{\mathcal{F}}$ metrizes the weak topology, where $\|\mathbb{P}\|_{\mathcal{F}}:=\sup\{\int f\,\mathrm{d}\mathbb{P}: f\in\mathcal{F}\}$. In this paper, we investigate this problem of optimal estimation in weak and strong topologies by choosing $\mathcal{F}$ to be a unit ball in a reproducing kernel Hilbert space (say $\mathcal{F}_H$ defined over $\mathbb{R}^d$), where this choice is both of theoretical and computational interest. Under some mild conditions on the reproducing kernel, we show that $\|\cdot\|_{\mathcal{F}_H}$ metrizes the weak topology and the kernel density estimator (with $L^1$ optimal bandwidth) estimates $\mathbb{P}$ at dimension independent optimal rate of $n^{-1/2}$ in $\|\cdot\|_{\mathcal{F}_H}$ along with providing a uniform central limit theorem for the kernel density estimator.

Subfields: Applied Mathematics, Numerical Analysis, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Mathematics

In Numerical Analysis

In Statistics and Probability

Citations in the corpus

Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, background): “While various estimators have been studied for µP (Muandet et al. 2016, Sriperumbudur 2016), Tolstikhin et al. (2016) recently showed that this rate is minimax optimal and so the empirical estimator µ̂P is a minimax optimal estimator of µP.”
Minimax Estimation of Kernel Mean Embeddings (2016, methodology): “We refer the reader to (Sriperumbudur et al., 2010; Sriperumbudur, 2016) for more details on the properties of μk and the corresponding MMD.”
Multi-fidelity Bayesian Optimisation with Continuous Approximations (2017, methodology): “Since ﬁnite dimensional kernels are typically not radial (Sriperumbudur et al., 2016), our analysis techniques will not carry over straightforwardly.”
Kernel Distribution Embeddings: Universal Kernels, Characteristic Kernels and Kernel Metrics on Distributions (2016, methodology): “They gave new suﬃcient conditions to metrize the weak convergence, which then got improved by Sriperumbudur (2016)[Thm. 2].”
Measuring Sample Quality with Kernels (2017, background): “Since r ( x ) (cid:44) exp −(cid:107) x / 2 is bounded and continuous and never vanishes, the kernel ˜ k b ( x, By (Sriperumbudur, 2016, Thm. 3.2), since ˜ k b is ISPD with ˜ k b ( x, · ) ∈ C 0 ( R d ) for all x , we know that d H ( µ m , P ) → 0 only if µ m ⇒ P .”
Two-stage sampled learning theory on distributions (2014, background): “e Polishness of X guarantees that M+ 1 (X) with the weak* topology is Polish (Steinwart & Christmann, 2008). Since X is LCH, and k is c 0-universal, the mean embedding induced topology coincides (Sriperumbudur, 2013) with the weak* topology on M+ 1 (X). Thus, X with the D(µa,µb) = kµa −µbkH metr”