EXACT AND ASYMPTOTICALLY ROBUST PERMUTATION TESTS

By E. Chung, Joseph P. Romano. 2013

264 citations recorded by the source corpus.

Given independent samples from P and Q, two-sample permutation tests allow one to construct exact level tests when the null hypothesis is P=Q. On the other hand, when comparing or testing particular parameters $\theta$ of P and Q, such as their means or medians, permutation tests need not be level $\alpha$, or even approximately level $\alpha$ in large samples. Under very weak assumptions for comparing estimators, we provide a general test procedure whereby the asymptotic validity of the permutation test holds while retaining the exact rejection probability $\alpha$ in finite samples when the underlying distributions are identical. The ideas are broadly applicable and special attention is given to the k-sample problem of comparing general parameters, whereby a permutation test is constructed which is exact level $\alpha$ under the hypothesis of identical distributions, but has asymptotic rejection probability $\alpha$ under the more general null hypothesis of equality of parameters. A Monte Carlo simulation study is performed as well. A quite general theory is possible based on a coupling construction, as well as a key contiguity argument for the multinomial and multivariate hypergeometric distributions.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/17198251047e920e02e8fc62cda8981f4fb343ba

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Covariate-adjusted Fisher randomization tests for the average treatment effect (2020, background): “Chung and Romano (2013) and Wu and Ding (2020) showed that randomization tests with appropriately studentized statistics are asymptotically valid for H0n.”
The Generalized Oaxaca-Blinder Estimator (2020, background): “See Section 4 in Cohen and Fogarty (2020) for an example.”
Minimax optimality of permutation tests (2020, background): “Hodara and Reynaud-Bouret (2019) present two exponential concentration inequalities for TRad based on the coupling argument introduced by Chung and Romano (2013).”
Robust Permutation Tests For Correlation And Regression Coefficients (2017, background): “For example, we will consider the case where k = p = 2, μ is zero, Σ = diag(2, 1), and r ∼ exp(1).”
Randomization Tests for Weak Null Hypotheses in Randomized Experiments (2018, methodology): “lem of un-studentized statistics in two sample tests. Janssen (1997) proposed using a studentized statistic to control the type I error without assuming equal distributions in the two sample problem. Chung and Romano (2013) studied the same phenomenon when the parameter of interest could be more gen”
Asymptotically valid and exact permutation tests based on two-sample U-statistics (2016, methodology): “The studentized permutation median test (Chung and Romano, 2011) results in a p-value of 0.”
Asymptotic permutation tests in general factorial designs (2015, methodology): “(d) In the one-way layout our permutation procedure coincides (except for their slightly stronger sample size conditions) with the recently suggested k-sample Behrens–Fisher procedure from Chung and Romano (2013); see example 3.1 and theorem 3.1 therein.”
Robust multivariate nonparametric tests via projection averaging (2018, methodology): “Hoeffding (1952) established a sufficient condition (indeed the necessary condition proved by Chung and Romano, 2013) under which the permutation distribution approximates the corresponding unconditional distribution. The condition is stated as follows: Lemma B.4 (Theorem 5.1 of Chung and Romano (20”
A paradox from randomization-based causal inference (2014, methodology): “In the permutation test literature, some authors (e.g., Neuhaus, 1993; Janssen, 1997; Chung and Romano, 2013; Pauly, Brunner, and Konietschke, 2015) suggested using the Studentized version of τ̂ , i.e., τ̂ /V̂ 1/2 Neyman, as the test statistic.”
A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification (2021, background): “…are other lines of work in statistics with equal claim to the term “distribution-free” especially when it is interpreted asymptotically, such as permutation tests [20], quantile regression [8], rank tests [21–23], and even the bootstrap [24, 25]— the following is not a history of those topics .”
Selective inference with a randomized response (2015, background): “Associated with μn is our statistic T which is a linearizable statistic [Chung and Romano (2013)].”
Multivariate and multiple permutation tests (2016, methodology): “Chung and Romano (2013) provide very general asymptotic arguments to handle general univariate testing problems.”
A Note on Conditional Versus Joint Unconditional Weak Convergence in Bootstrap Consistency Results (2017, background): “Parts of these equivalences can also be found in Dümbgen and Del Conte-Zerial (2013), Lemma 4.1, relying on ideas put forward in Hoeffding (1952) and also exploited in Romano (1989) and Chung and Romano (2013).”
Agnostic notes on regression adjustments to experimental data: Reexamining Freedman's critique (2012, background): “Related issues appear in discussions of the two-sample problem [Miller (1986), pp. 56–62, Stonehouse and Forrester (1998)] and randomization tests [Gail et al. (1996), Chung and Romano (2011a, 2011b)].”