A Hilbert Space Embedding for Distributions

By Alex Smola, Arthur Gretton, Le Song, Bernhard Schölkopf. 2007

1,003 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Probabilistic & Bayesian Methods

Citations in the corpus

Improved Techniques for Training GANs (2016, neutral): “3.1, is similar in spirit to approaches that use maximum mean discrepancy [7, 8, 9] to train generator networks [10, 11].”
Discriminative Embeddings of Latent Variable Models for Structured Data (2016, positive): “Our idea is to model each structured data point as a latent variable model, then embed the graphical model into feature spaces (Smola et al., 2007; Song et al., 2009), and use inner product in the embedding space to define kernels.”
GAIT: A Geometric Approach to Information Theory (2019, methodology): “The operator KP corresponds to the embedding of the space of distributions on X into a reproducing kernel Hilbert space used for comparing distributions without the need for density estimation (Smola et al., 2007).”
Sequential Kernel Herding: Frank-Wolfe Optimization for Particle Filtering (2015, methodology): “It turns out that the worst-case error of estimators of the form (2) can be analyzed in terms of their approximation distance to the mean element µ ( p ) := E p [Φ] ∈ H (Smola et al., 2007; Sriperumbudur et al., 2010).”
On the Equivalence between Herding and Conditional Gradient Algorithms (2012, positive): “Following Chen et al. (2010), we denote by µ the mean element (see, e.g., Smola et al., 2007): Note that in the learning perspective, p is the empirical distribution on the data and so µ is the corresponding empirical moment vector to match.”
Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, methodology): “Detailed exposition on RKHSs can be found in Schölkopf and Smola (2002) and Berlinet and Thomas-Agnan (2004), for example.”
Discovering Causal Signals in Images (2016, methodology): “Since the input patterns Si are not fixed-dimensional vectors, but bags of points, we borrow inspiration from the literature on kernel mean embedding classifiers [28] and construct a feedforward neural network of the form”
Domain Adaptation with Conditional Transferable Components (2016, methodology): “Finally, we present a computationally efﬁcient method to estimate the involved parameters based on kernel mean embedding of distributions (Smola et al., 2007; Gretton et al., 2012).”
Kernel‐based tests for joint independence (2016, methodology): “This work extends the two-variable HSIC (Gretton et al., 2005, 2007; Smola et al., 2007) to testing joint independence for an arbitrary number of variables.”
Multi-Source Domain Adaptation: A Causal View (2015, methodology): “1.d, we know that P X | Y = c j , or computationally more easily, its kernel embedding (Smola et al. 2007; Gretton et al. 2007), is actually a function of V s : where F j are inﬁnite-dimensional vector functions, which might vary for different values of j .”
Prototypal Analysis and Prototypal Regression (2017, background): “2 Prototypal Learning on Distributions through Kernel Embedding Probability distributions or samples thereof can also be mapped to a reproducing kernel Hilbert space via kernel embedding (see Berlinet and Thomas-Agnan, 2004; Gretton et al., 2006; Smola et al., 2007; Sriperumbudur et al., 2010; Sejdi”
Consistent Kernel Mean Estimation for Functions of Random Variables (2016, methodology): “Q, µ̂1 is √ N -consistent (Smola et al., 2007).”
Interpretable Distribution Features with Maximum Testing Power (2016, background): “One can intuitively think of the ME test statistic as a squared normalized (by the inverse covariance S−1 n ) L (2)(X , VJ) distance of the mean embeddings (Smola et al., 2007) of the empirical measures Pn := 1 n ∑n i=1 δxi , and Qn := 1 n ∑n i=1 δyi where VJ := 1 J ∑J i=1 δvi , and δx is the Dirac ”
Minimax Estimation of Kernel Mean Embeddings (2016, background): “…(Gretton et al., 2007, 2012), kernel independence and conditional independence tests (Gretton et al., 2008; Fukumizu et al., 2008), covariate-shift (Smola et al., 2007), density estimation (Sriperumbudur, 2011), feature selection (Song et al., 2012), causal inference (Lopez-Paz et al., 2015),…”
Kernel Sequential Monte Carlo (2015, methodology): “e estimate might result in poor MH rejuvenation moves. First, a Gaussian measure is ﬁtted to the particle embeddings in the RKHS using their empirical mean embedding and empirical covariance operator [21]. Even though the RKHS can be inﬁnitely-dimensional (and in many interesting cases including Gau”
Kernel-Based Just-In-Time Learning for Passing Expectation Propagation Messages (2015, methodology): “We may embed individual probability distributions to these RKHSs, following (Smola et al., 2007).”
On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions (2015, methodology): “R , n , which can be computed in closed form for several triplet ( k, g, dρ ) (see, e.g., Smola et al., 2007; Oates and Girolami, 2015), then the problem above is equivalent to minimizing with respect to β : which leads to a n × n linear system with running time complexity O ( n 3 ) .”
K2-ABC: Approximate Bayesian Computation with Infinite Dimensional Summary Statistics via Kernel Embeddings (2015, methodology): “For a probability distribution Fx on a domain X , its kernel embedding is defined as µFx = EX∼Fxk(·, X) (Smola et al., 2007), an element of an RKHS H associated with a positive definite kernel k : X × X → R.”
Computing functions of random variables via reproducing kernel Hilbert space representations (2015, neutral): “We summarize the known results (Stein-wart and Christmann 2008; Fukumizu et al. 2008; Smola et al. 2007; Gretton et al. 2012; Sriperumbudur et al. 2010) in Tables 1 and 2.”