Kernel‐based tests for joint independence

By Niklas Pfister, Peter Buhlmann, Bernhard Schölkopf, Jan Peters. 2016

224 citations recorded by the source corpus.

We investigate the problem of testing whether d possibly multivariate random variables, which may or may not be continuous, are jointly (or mutually) independent. Our method builds on ideas of the two‐variable Hilbert–Schmidt independence criterion but allows for an arbitrary number of variables. We embed the joint distribution and the product of the marginals in a reproducing kernel Hilbert space and define the d‐variable Hilbert–Schmidt independence criterion dHSIC as the squared distance between the embeddings. In the population case, the value of dHSIC is 0 if and only if the d variables are jointly independent, as long as the kernel is characteristic. On the basis of an empirical estimate of dHSIC, we investigate three non‐parametric hypothesis tests: a permutation test, a bootstrap analogue and a procedure based on a gamma approximation. We apply non‐parametric independence testing to a problem in causal discovery and illustrate the new methods on simulated and real data sets.

Subfields: Artificial Intelligence, Statistics and Probability, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Statistical Learning Theory

Citations in the corpus

Toward Causal Representation Learning (2021, background): “Recent progress in (conditional) independence testing heavily relies on kernel function classes to represent probability distributions in reproducing kernel Hilbert spaces [43], [61], [74], [91], [92], [193], [280].”
Towards Causal Representation Learning (2021, positive): “progress in (conditional) independence testing heavily relies on kernel function classes to represent probability distributions in reproducing kernel Hilbert spaces [90, 91, 73, 278, 60, 191, 42].”
Causality for Machine Learning (2019, methodology): “Recent progress in (conditional) independence testing heavily relies on kernel function classes to represent probability distributions in reproducing kernel Hilbert spaces (Gretton et al., 2005a,b; Fukumizu et al., 2008; Zhang et al., 2011; Pfister et al., 2018b; Chalupka et al., 2018).”
Invariant Causal Prediction for Sequential Data (2017, methodology): “An implementation of the Hilbert-Schmidt independence criterion is given in the R-package dHSIC (Pfister et al., 2017).”
Testing mutual independence in high dimension via distance covariance (2016, methodology): “In comparison, dHSIC proposed by Pfister et al. (2017) targets the joint (mutual) dependence.”
A New Coefficient of Correlation (2019, background): “There have been many proposals to address this deficiency of the classical coefficients [20], such as the maximal correlation coefficient [5, 12, 18, 29], various coefficients based on joint cumulative distribution functions and ranks [3, 4, 7–9, 15, 19, 26, 28, 32, 33, 43–45], kernel-based methods ”
Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences (2018, background): “, 2018], and causal discovery [Pfister et al., 2017], to name a few.”
The conditional permutation test (2018, background): “Simulta-neously, a great deal of research eﬀort has gone into developing several diﬀerent tests of independence, for example based on ranks, kernel methods, copulas, and nearest neighbours [26, 13, 12, 4].”
Deep Mixture Generative Autoencoders (2021, background): “The deﬁnition of the cross-covariance operator can be extended for assessing the independence of d variables, similar to the expressions from (13) and (14), representing the latent space { z 1 , z 2 , . . . , z d } deﬁned by d VAE components [20], [38].”
Statistical Dependence: Beyond Pearson’s ρ (2018, background): “(2013), and Pfister et al. (2018). We have found the quite early paper by Gretton et al.”
Information Constraints on Auto-Encoding Variational Bayes (2018, background): “The dHSIC [18, 19] generalizes the HSIC to d variables.”
High-dimensional consistent independence testing with maxima of rank correlations (2018, methodology): “Recent work thus proposed the use of consistent rank (Bergsma and Dassios, 2014), kernel-based (Gretton et al., 2008; Pfister et al., 2018), and distance covariance/correlation statistics (Székely et al., 2007).”
The conditional permutation test for independence while controlling for confounders (2018, methodology): “of independence, for example based on ranks, kernel methods, copulas, and nearest neighbours [32, 17, 16, 6].”
A fast algorithm for computing distance correlation (2018, methodology): “HSIC has been extensively studied in machine learning literature (Gretton et al., 2005, 2008; Pfister et al., 2018).”
Mixtures of Variational Autoencoders (2020, background): “Similarly to (12), the definition of the cross-covariance operator can be extended into the dHSIC criterion for assessing the independence of d variables, [16], [17].”