Universality, Characteristic Kernels and RKHS Embedding of Measures

By Bharath K. Sriperumbudur, Kenji Fukumizu, Gert R. G. Lanckriet. 2010

600 citations recorded by the source corpus.

Over the last few years, two different notions of positive definite (pd) kernels---universal and characteristic---have been developing in parallel in machine learning: universal kernels are proposed in the context of achieving the Bayes risk by kernel-based classification/regression algorithms while characteristic kernels are introduced in the context of distinguishing probability measures by embedding them into a reproducing kernel Hilbert space (RKHS). However, the relation between these two notions is not well understood. The main contribution of this paper is to clarify the relation between universal and characteristic kernels by presenting a unifying study relating them to RKHS embedding of measures, in addition to clarifying their relation to other common notions of strictly pd, conditionally strictly pd and integrally strictly pd kernels. For radial kernels on ℜd, all these notions are shown to be equivalent.

Subfields: Artificial Intelligence, Control and Systems Engineering, Statistical and Nonlinear Physics, Probabilistic & Bayesian Methods, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Statistical and Nonlinear Physics

In Probabilistic & Bayesian Methods

In Statistical Learning Theory

Citations in the corpus

Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, methodology): “This survey will deal mostly with the embedding of distributions in a Hilbert space, while extensions to Banach spaces are investigated in Zhang et al. (2009) and Sriperumbudur et al. (2011b).”
Domain Adaptation with Conditional Transferable Components (2016, methodology): “In this paper, we adopt the Gaussian kernel function, which has been shown to be characteristic (Sriperumbudur et al., 2011).”
Balanced Policy Evaluation and Learning (2017, background): “In lieu of specification, consistency is also guaranteed if the RKHS product consists of C0-universal kernels, defined below, such as the RBF kernel [40].”
Consistent Kernel Mean Estimation for Functions of Random Variables (2016, methodology): “mple guarantees for the estimator ^ f(X) deﬁned in (1). They are based on the convergence of ^ X and avoid simplifying assumptions about its structure. 4 2.1 Consistency If k xis c 0-universal (see Sriperumbudur et al. (2011)), consistency of ^ f(X) can be shown in a rather general setting. Theorem ”
Interpretable Distribution Features with Maximum Testing Power (2016, background): “It follows that under H 1 , assuming that k is characteristic (Sriperumbudur et al., 2011), we have λ n > 0 .”
Bayesian Learning of Kernel Embeddings (2016, background): “For a broad family of kernels termed characteristic (Sriperumbudur et al., 2011), every probability measure has a unique embedding – thus, such embeddings completely determine their probability measures and capture all of the moment information.”
Minimax Estimation of Kernel Mean Embeddings (2016, background): “oning will work as long as ψis not “ﬂat” on the ball of radius Rcentered around origin. 5.2 Proof of Corollary 2 The proof is based on application of Theorem 1. Since supp(ν) 6= {0}, it follows from (Sriperumbudur et al., 2011, Proposition 5) that kis characteristic. We now show that there exist z∈ ”
Fast Two-Sample Testing with Analytic Representations of Probability Measures (2015, background): “Traditional approaches to two-sample testing are based on distances between representations of the distributions, such as density functions, cumulative distribution functions, characteristic functions or mean embeddings in a reproducing kernel Hilbert space (RKHS) [26, 25].”
K2-ABC: Approximate Bayesian Computation with Infinite Dimensional Summary Statistics via Kernel Embeddings (2015, methodology): “While simple kernels like polynomial of order r capture differences in first r moments of distributions, particularly interesting are kernels with a characteristic property4(Sriperumbudur et al., 2011), for which the kernel embedding is injective and thus MMD defined by such kernels gives a metric o”
Kernel Mean Shrinkage Estimators (2014, background): “By invoking Bochner’s theorem, which states that ψ is the Fourier transform of a non-negative ﬁnite Borel measure Λ, i.e. , ψ ( x ) = e − ix ⊤ ω dΛ( ω ) , x ∈ R d , we obtain (see (16) in the proof of Proposition 5 in Sriperumbudur et al. (2011)) where φ P is the characteristic function of P .”
A Kernel Independence Test for Random Processes (2014, background): “For k and l characteristic, continuous, translation invariant, and vanishing at infinity, γ is equal to zero if and only if the null hypothesis holds (see (Lyons, 2013, Lemma 3.8), applying (Sriperumbudur et al., 2011, Proposition 2), and the note at the end of Section 5).”
Density Estimation in Infinite Dimensional Exponential Families (2013, methodology): “[33, Proposition 5] shows that H is dense in C 0 (Ω) w.r.t. uniform norm if and only if k satisﬁes (5).”
Kernel Adaptive Metropolis-Hastings (2013, background): “Such kernels are said to becharacteristic (Sriperumbudur et al., 2010; 2011), since each distribution is uniquely characterized by its embedding (in the same way that every probability distribution has a unique characteristic function).”