Tailoring density estimation via reproducing kernel moment matching

By Le Song, Xinhua Zhang, Alex Smola, Arthur Gretton, Bernhard Schölkopf. 2008

65 citations recorded by the source corpus.

Moment matching is a popular means of parametric density estimation. We extend this technique to nonparametric estimation of mixture models. Our approach works by embedding distributions into a reproducing kernel Hilbert space, and performing moment matching in that space. This allows us to tailor density estimators to a function class of interest (i.e., for which we would like to compute expectations). We show our density estimation approach is useful in applications such as message compression in graphical models, and image classification and retrieval.

Subfields: Artificial Intelligence, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Probabilistic & Bayesian Methods

Citations in the corpus

Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, methodology): “The MMD has been applied extensively in many applications, namely, clustering (Jegelka et al. 2009), density estimation (Song et al. 2007a; 2008), (conditional) independence tests (Fukumizu et al. 2008, Doran et al. 2014, Chwialkowski and Gretton 2014), causal discovery (Sgouritsa et al. 2013, Chen…”
Computing functions of random variables via reproducing kernel Hilbert space representations (2015, positive): “Alternatively, if p has a density, we can estimate it using methods such as reproducing kernel moment matching and combinations with kernel density estimation (Song et al. 2008; Kanagawa and Fukumizu 2014).”
Kernel Mean Estimation via Spectral Filtering (2014, methodology): “We apply Spectral-KMSE to the density estimation problem vi a kernel mean matching [1, 26].”
Characteristic Kernels and Infinitely Divisible Distributions (2014, methodology): “Density Estimation: (Smola et al., 2007; Song et al., 2008; McCalman et al., 2013) Let m[P ] and m[p] denote the kernel mean of a probability distribution P and density p in an RKHS H, respectively.”
Kernel Mean Estimation and Stein's Effect (2013, positive): “We consider three benchmark applications: density estimation via kernel mean matching [20], kernel PCA using shrinkage mean and covariance operator [2], and discriminative learning on distributions [25, 26].”
Optimally-Weighted Herding is Bayesian Quadrature (2012, background): “Whilst the integration can indeed be carried out analytically in several cases (Song et al., 2008; Chen et al., 2010), this requirement is the most pertinent limitation on applications of kernel herding, Bayesian quadrature and related algorithms.”
A Kernel Method for the Two-Sample-Problem (2006, background): “We also demonstrate excellent performance when comparing distributions over graphs, for which no alternative tests currently exist.”
Robust kernel density estimation (2008, methodology): “Song et al. (2008) adopt a different criterion based on Hilbert space embeddings of probability distributions.”
Recovering Distributions from Gaussian RKHS Embeddings (2014, methodology): “On the other hand, a method for estimating the density of P from m̂P has been proposed, assuming that P is a Gaussian mixture (Song et al., 2008; McCalman et al., 2013).”
An Empirical Analysis of Domain Adaptation Algorithms for Genomic Sequence Analysis (2008, background): “It is often not necessary to retain all information (indeed, it may be useful to specify which information we want to retain and which one we want to disregard, see [8]).”