Research Commentary - Too Big to Fail: Large Samples and the p-Value Problem

By Mingfeng Lin, Henry C. Lucas Jr., Galit Shmueli. Information systems research, 2013

930 citations recorded by the source corpus.

Subfields: Information Systems and Management, Management Information Systems, Sociology and Political Science

View Paper: https://www.semanticscholar.org/paper/7241c748932deb734fff1681e951e50be0853a39

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems and Management

1. Christopher T. Lowenkamp
2. J. Skeem
3. Patrick Mikalef
4. John Krogstie
5. Maryam Ghasemaghaei
6. Goran Calic

In Management Information Systems

1. Suprateek Sarker
2. Roger H. L. Chiang
3. Christopher T. Lowenkamp
4. J. Skeem

In Sociology and Political Science

1. J. Skeem
2. Christopher T. Lowenkamp
3. Suprateek Sarker
4. Ahmed Ali Abbasi
5. Roger H. L. Chiang
6. Alexandra Chouldechova
7. John Krogstie
8. Patrick Mikalef

Citations in the corpus

• Real-world menstrual cycle characteristics of more than 600,000 menstrual cycles (2019, methodology): “Owing to the very large sample sizes in this study, P values were not calculated since they can be very small even if differences between cohorts are of no clinical significance.(49) Instead, effect size between two cohorts was estimated as a mean difference with a 95% confidence interval calculated”
• RISK, RACE, AND RECIDIVISM: PREDICTIVE BIAS AND DISPARATE IMPACT*: RISK, RACE, AND RECIDIVISM (2016, methodology): “Given that even trivially small differences can become statistically significant in samples as large as ours (Lin, Lucas, and Shmueli, 2013), we use an alpha level of .001 to signal statistical significance and focus on effect sizes in interpreting results.”
• Sample size, power and effect size revisited: simplified and practical approaches in pre-clinical, clinical and laboratory studies (2020, neutral): “tions could be utilized for statistical evaluations of the big data results (18,104).”
• Big Data Research in Information Systems: Toward an Inclusive Research Agenda (2016, background): “In addition to statistical significance and co-efficient signs, one may also need to consider effect sizes and variance when testing hypotheses on big data sets (Lin et al., 2013; George et al., 2014).”
• Multigroup Analysis using SmartPLS: Step-by-Step Guidelines for Business Research (2020, neutral): “As p-values more easily obtain significant results due to large sample sizes (see Lin et al., 2013), it is often considered a “rough yardstick” to reject a model or hypothesis (Hahn, 2016).”