Kernel Bayes' Rule

By Kenji Fukumizu, Le Song, Arthur Gretton. 2010

65 citations recorded by the source corpus.

A nonparametric kernel-based method for realizing Bayes' rule is proposed, based on kernel representations of probabilities in reproducing kernel Hilbert spaces. The prior and conditional probabilities are expressed as empirical kernel mean and covariance operators, respectively, and the kernel mean of the posterior distribution is computed in the form of a weighted sample. The kernel Bayes' rule can be applied to a wide variety of Bayesian inference problems: we demonstrate Bayesian computation without likelihood, and filtering with a nonparametric state-space model. A consistency rate for the posterior estimate is established.

Subfields: Probabilistic & Bayesian Methods, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

In Statistical Learning Theory

Citations in the corpus

Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, methodology): “…these operations in non-parametric probabilistic inference, e.g., filtering for dynamical systems (Song et al. 2009), kernel belief propagation (Song et al. 2011a), kernel Monte Carlo filter (Kanagawa et al. 2013), kernel Bayes’ rule (Fukumizu et al. 2011), often with strong theoretical guarantees.”
Kernel Mean Shrinkage Estimators (2014, methodology): “…appealing to many algorithms in modern kernel methods, namely, two-sample testing via maximum mean discrepancy (MMD) (Gretton et al., 2007, 2012), kernel independence tests (Gretton et al., 2008), Hilbert space embedding of HMMs (Song et al., 2010), and kernel Bayes rule (Fukumizu et al., 2011).”
Characteristic Kernels and Infinitely Divisible Distributions (2014, methodology): “(6) The given kernel mean estimator m̂[P ] often has a weighted sum form m̂[P ] = ∑n i=1 wik(·,Xi), with data {Xi} and weight vector w ∈ Rn (Song et al., 2009; Fukumizu et al., 2011).”
Filtering with State-Observation Examples via Kernel Monte Carlo Filter (2013, methodology): “te transitions are also given, and the transition model is to be learned nonparametrically from these examples. For this setting, there are methods based on kernel mean embeddings (Song et al., 2009; Fukumizu et al., 2011, 2013) and Gaussian Processes (Ko and Fox, 2009; Deisenroth et al., 2009). The”
Hilbert Space Embeddings of Predictive State Representations (2013, methodology): “and C XY represent the distribution P[X;Y]. One is dened as an element of F G, and the other as a linear operator from Gto F, but they are isomorphic under the standard identication of these spaces [9], so we abuse notation and write XY= C . Given T i.i.d. pairs of observations (x t;y t), dene X = ”
Kernel Mean Estimation and Stein's Effect (2013, positive): “As a result, there have been successful applications that benefit from the kernel mean representation, namely, a maximum mean discrepancy (MMD) [8], kernel dependency measure [9], kernel two-sample-test [10], Hilbert space embedding of HMMs [11], and kernel Bayes rule [12].”
Conditional mean embeddings as regressors (2012, background): “Deﬁne the uncentred covariance operators, where the tensor product satisﬁes ( a ⊗ b ) c = (cid:104) b, c (Fukumizu et al., 2011, Appendix), boundedness should be sufﬁcient, although this is outside the scope of the present paper).”
Hypothesis testing using pairwise distances and associated kernels (2012, methodology): “…the associated kernels may be used much more widely: in conditional dependence testing and estimates of the chi-squared distance (Fukumizu et al., 2008), in Bayesian inference (Fukumizu et al., 2011), in mixture density estimation (Sriperumbudur, 2011) and in other machine learning applications.”
Learning predictive models of a depth camera & manipulator from raw execution traces (2014, methodology): “One is defined as an element of F⌦G, and the other as a linear operator from G to F , but they are isomorphic under the standard identification of these spaces [34], so we abuse notation and write μXY = CXY .”
Path Integral Control by Reproducing Kernel Hilbert Space Embedding (2012, background): “, 2009; 2011] and [Fukumizu et al., 2011] provide the basic concepts on which we build.”
Kernel Embeddings of Conditional Distributions: A Unified Kernel Framework for Nonparametric Inference in Graphical Models (2013, methodology): “In this paper, we will review the recent development of nonparametric inference with kernel embeddings of conditional distributions [4, 5, 6, 7, 8, 9, 10, 11, 12], and provide a novel, unified kernel framework for addressing a broad class of challenging problems in graphical models.”
A Generalized Kernel Approach to Structured Output Learning (2012, result): “The idea here is similar to the one used by Fukumizu et al. (2011).”
Smooth Operators (2013, methodology): “…include the sum rule for marginalising out a random variable in a multivariate distribution (Song, Huang, Smola, and Fukumizu, 2009), and kernel-based approximations to Bayes' rule for inference without parametric models (Fukumizu, Song, and Gretton, 2011; Song, Fukumizu, and Gretton, 2013).”
Monte Carlo Filtering Using Kernel Embedding of Distributions (2014, methodology): “Note that the consistency of the correction step is guaranteed by the convergence theorem of Kernel Bayes’ rule (Fukumizu, Song, and Gretton 2011; 2014).”
Kernel approximate Bayesian computation in population genetic inferences (2012, methodology): “We have (Fukumizu et al. 2010, 2011) 〈𝑓 ( ∙ ), 𝜇̂ 𝜃 | 𝑠 〉 ℋ − 𝐸 [ 𝑓 ( We want to establish �𝐶̂ �𝐶̂ + 𝜀 First we show �𝐶 �𝐶 + 𝜀 𝐼 � 𝑓 ( ∙ ) − 𝐶 ( 𝐶 + 𝜀 The left hand side is upper bounded by �𝐶̂ �𝐶̂ + 𝜀 𝐼 � �𝐶̂ − 𝐶 � ( 𝐶 𝑠𝑠 + 𝜀 𝐼 𝑛 ) 𝑓 ( ∙ ) � ℋ 𝜃 By the decomposition…”
Recovering Distributions from Gaussian RKHS Embeddings (2014, methodology): “Here, we define an interval in R d as Ω := [a 1 , b 1 ] × · · · [a d , b d ], where −∞ a i b i +∞, i = 1, . . . , d. Note that we may use the estimated P (Ω) for making credible intervals fromˆm P in Bayesian inference applications (Fukumizu et al., 2011).”