Testing mutual independence in high dimension via distance covariance

By Shu Yao, Xianyang Zhang, Xiaofeng Shao. 2016

99 citations recorded by the source corpus.

We introduce an L2 ‐type test for testing mutual independence and banded dependence structure for high dimensional data. The test is constructed on the basis of the pairwise distance covariance and it accounts for the non‐linear and non‐monotone dependences among the data, which cannot be fully captured by the existing tests based on either Pearson correlation or rank correlation. Our test can be conveniently implemented in practice as the limiting null distribution of the test statistic is shown to be standard normal. It exhibits excellent finite sample performance in our simulation studies even when the sample size is small albeit the dimension is high and is shown to identify non‐linear dependence in empirical data analysis successfully. On the theory side, asymptotic normality of our test statistic is shown under quite mild moment assumptions and with little restriction on the growth rate of the dimension as a function of sample size. As a demonstration of good power properties for our distance‐covariance‐based test, we further show that an infeasible version of our test statistic has the rate optimality in the class of Gaussian distributions with equal correlation.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/034b67fda453b940712331699285095fd87fe508

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Nonparametric independence testing via mutual information (2017, background): “This concept has also been studied in high dimensions (Sz´ekely and Rizzo, 2013; Yao, Zhang and Shao, 2017), and for testing independence of several random vectors (Fan et al., 2017).”
Statistical Dependence: Beyond Pearson’s ρ (2018, methodology): “The discrepancy between the two distributions has been analyzed by Zhang et al. (2018) and Yao, Zhang, and Shao (2018) Lately there have been other extensions of both the dcov and HSIC to conditional dependence, partial distance and to time series. A few references are Szekely and Rizzo (2014), Zhan”
High-dimensional consistent independence testing with maxima of rank correlations (2018, methodology): “To reflect this, we consider two sets of examples that focus on relatively sparse settings (modified based on Yao et al. (2018) and Han et al. (2017)) but also include a very dense third setup (modified based on Leung and Drton (2018) with an adjustment to dimension as suggested in Cai and Ma…”