Kernel Mean Estimation and Stein's Effect

By Krikamol Muandet, Kenji Fukumizu, Bharath K. Sriperumbudur, Arthur Gretton, Bernhard Schölkopf. 2013

41 citations recorded by the source corpus.

A mean function in reproducing kernel Hilbert space, or a kernel mean, is an important part of many applications ranging from kernel principal component analysis to Hilbert-space embedding of distributions. Given finite samples, an empirical average is the standard estimate for the true kernel mean. We show that this estimator can be improved via a well-known phenomenon in statistics called Stein's phenomenon. After consideration, our theoretical analysis reveals the existence of a wide class of estimators that are better than the standard. Focusing on a subset of this class, we propose efficient shrinkage estimators for the kernel mean. Empirical evaluations on several benchmark applications clearly demonstrate that the proposed estimators outperform the standard kernel mean estimator.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Statistics and Probability, Probabilistic & Bayesian Methods, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

In Statistics and Probability

In Probabilistic & Bayesian Methods

In Statistical Learning Theory

Citations in the corpus

Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, methodology): “Later, Muandet et al. (2014b) provides a non-linear extension of the estimator in (3.22) by means of spectral filtering algorithms (Bauer et al. 2007, De Vito et al. 2006).”
Bayesian Learning of Kernel Embeddings (2016, methodology): “(7) recovers the S-KMSE estimator (Muandet et al., 2013), where the regularization parameter is related to the variance in the likelihood model (5), with a difference that in our case the kernel kθ used to compute the empirical embedding is not the same as the kernel rθ used to compute the kernel ma”
Computing functions of random variables via reproducing kernel Hilbert space representations (2015, positive): “i.d. sample from p (cf. (10)). As an aside, note that using ideas from James-Stein estimation [17], we can construct shrinkage estimators that improve upon the standard empirical estimator (see e.g., [25, 26]). One can show that m[pˆ m] converges at rate m 1=2 (cf. [36] and [37, Theorem 27]): Theore”
Kernel Mean Estimation via Spectral Filtering (2014, methodology): “Previous work [1] has shown that shrinkage can help in constructing “better” estimators of t he kernel mean than the empirical estimator.”
Nonparametric Independence Testing for Small Sample Sizes (2014, methodology): “A first step was taken by [15], who extend their techniques for a more accurate kernel mean map to also mention two shrinkage estimators for cross-covariance operators.”
Kernel Mean Shrinkage Estimators (2014, methodology): “Muandet et al. (2014b) leverages this observation and generalizes S-KMSE to a family of shrinkage estimators via spectral ﬁltering algorithms.”
The Randomized Causation Coefficient (2014, background): “Then, we can approximate (1) by µ̂k(X) := 1 n n∑ i=1 k(xi, ·) ∈ Hk. (3) Though it can be improved (Muandet et al., 2014), the estimator (3) is the most common due to its ease of implementation.”
Who Supported Obama in 2012?: Ecological Inference through Distribution Regression (2015, methodology): “…separately to estimate: Next, use kernel ridge regression [29] to learn a function f : where the objective is to minimize the L 2 loss subject to a “ridge” complexity penalty weighted by a positive constant λ : In [33] a variety of kernels for f corresponding to the Hilbert space H f are…”