The analysis of 2 × 2 contingency tables—yet again: Author's reply

By Stian Lydersen, Petter Laake, Morten Wang Fagerland. 2011

54 citations recorded by the source corpus.

We thank John Richardson for taking an interest in our recent article [1]. Richardson finds that we ought to have mentioned the ‘N −1’ chi-squared test as a recommended method for testing association in 2×2 tables. However, a large number of tests have been suggested in the literature. As an example, Martín Andrés and Silva Mato [2] studied 60 asymptotic tests for comparing binomial proportions. In our paper, we described only the most commonly used tests and some lesser known tests with excellent performance. We are well aware of the ‘N −1’ chi-squared test, as well as the adapted procedure recommended by Campbell [3]. The ‘N −1’ chi-squared test was one of the five tests compared by Hirji et al. [4]. Over a wide range of sample sizes, the ‘N −1’ chi-squared test was shown to violate the nominal significance level more often than the exact conditional mid p test. For example, for a two-sided test with nominal significance level 0.05, the maximum actual significance level was 0.128 for the ‘N −1’ chi-squared test, and 0.057 for the twice the smallest tail mid p test. We strongly agree with the recommendations of Hirji et al. [4]: the exact mid p test is preferable to the ‘N −1’ chi-squared test. Campbell’s adaption [3] is as follows: Use the ‘N −1’ chi-squared test unless the smallest expected count is less than 1, else, use Fisher–Irwin’s exact test. The actual significance level of the resulting test procedure still exceeds the nominal level, but the difference is greatly reduced, giving a test with improved performance. Figures 1–4 in [3] show that Campbell’s procedure violates the nominal significance level more frequently than the exact mid p test (version twice the smallest tail). Hence, it is likely that Campbell’s procedure tends to have higher power at the cost of more frequently exceeding the nominal level. We prefer the exact mid p test over Campbell’s procedure. The exact mid p test works well for all distributions of counts, without the need for a separate method when an expected count is low. Furthermore, the mid p approach in general has many attractive properties (Hirji [5, p. 218]). Richardson claims that our recommendations are ‘technically complex methods’. We do not agree. The exact conditional mid p value can easily be calculated using practically any software that supports Fisher’s exact test. If it is required never to exceed the nominal significance level, our recommended exact unconditional p value can easily be calculated using both commercial and free software [1]. None of the methods suggested in Richardson’s letter preserve the nominal significance level.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/c9008d39cf544c6c61e8db6cd068a73dea8dcf17