A paradox from randomization-based causal inference

By Peng Ding. 2014

81 citations recorded by the source corpus.

Under the potential outcomes framework, causal effects are defined as comparisons between potential outcomes under treatment and control. To infer causal effects from randomized experiments, Neyman proposed to test the null hypothesis of zero average causal effect (Neyman's null), and Fisher proposed to test the null hypothesis of zero individual causal effect (Fisher's null). Although the subtle difference between Neyman's null and Fisher's null has caused lots of controversies and confusions for both theoretical and practical statisticians, a careful comparison between the two approaches has been lacking in the literature for more than eighty years. We fill in this historical gap by making a theoretical comparison between them and highlighting an intriguing paradox that has not been recognized by previous researchers. Logically, Fisher's null implies Neyman's null. It is therefore surprising that, in actual completely randomized experiments, rejection of Neyman's null does not imply rejection of Fisher's null for many realistic situations, including the case with constant causal effect. Furthermore, we show that this paradox also exists in other commonly-used experiments, such as stratified experiments, matched-pair experiments, and factorial experiments. Asymptotic analyses, numerical examples, and real data examples all support this surprising phenomenon. Besides its historical and theoretical importance, this paradox also leads to useful practical implications for modern researchers.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/ffc14b44ffb1b26416c8d83608f48ff54bdb2954

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Bridging Finite and Super Population Causal Inference (2017, background): “…luke_miratrix@gse.harvard.edu 2002; Imai 2008; Freedman 2008a,b; Rosenbaum 2010; Aronow and Middleton 2013; Aronow et al. 2014; Abadie et al. 2014; Miratrix et al. 2013; Ding 2016; Lin 2013; Middleton and Aronow 2015; Imbens and Rubin 2015; Chiba 2015; Rigdon and Hudgens 2015; Li and Ding 2016).”
Regression‐assisted inference for the average treatment effect in paired experiments (2016, background): “ standard normal cumulative distribution function. This normal approximation is justiﬁed under regularity conditions on the sequence of constants {(ϕi1,ϕi2),i = 1,...,n} as n → ∞; see Imai (2008) and Ding (2016) among many for more details. 3 4 Regression assisted estimators 4.1 Two intercepts from ”
Causal inference from 2K factorial designs by using potential outcomes (2015, background): “An intuitive justification of this can be found in Ding (2014), which compares the frequentist properties of the Neymanian sampling intervals and the Fisherian intervals and shows that the estimated Neymanian variance of an estimated factorial effect is asymptotically smaller than the estimated…”
Panel experiments and dynamic causal effects: A finite population perspective (2020, background): “, Hall and Heyde (1980)) through time and conditionally independent over the crosssection.”
Sensitivity analysis for inverse probability weighting estimators via the percentile bootstrap (2017, background): “The distinction can be best understood by comparing to the distinction between hypothesis testing and point estimation, or in Ding (2017)’s terminology, the subtle difference between Neyman’s null (the average causal effect is zero) and Fisher’s null (the individual causal effects are all zero). • E”
Randomization Tests for Weak Null Hypotheses in Randomized Experiments (2018, background): “Below are our principal asymptotic tools, which are consequences of Li and Ding (2017).”
Econometric Methods for Program Evaluation (2018, background): “While there is some very recent work that addresses the issue of extrapolation of RD treatment effects (e.g., Angrist and Rokkanen, 2015; Dong and Lewbel, 2015; Bertanha and Imbens, 2017; Cattaneo et al., 2017f), more work is certainly needed, covering both methodological and empirical issues.”
General Forms of Finite Population Central Limit Theorems with Applications to Causal Inference (2016, methodology): “…because the treatment and control groups are simple random samples from the finite experimental units, the CLTs for sampling surveys are sometimes adequate for establishing asymptotic distributions of the causal effect estimators (e.g., Liu and Hudgens 2014; Ding 2017; Ding and Dasgupta 2016).”