The Cult of Statistical Significance
By S. Ziliak, Deirdre Mccloskey. 2009
291 citations recorded by the source corpus.
We want to persuade you of one claim: that William Sealy Gosset (1876-1937)—aka "Student" of "Student's" t-test—was right, and that his difficult friend, Ronald A. Fisher (1890-1962), though a genius, was wrong. Fit is not the same thing as importance. Statistical significance is not the same thing as scientific importance or economic sense. But the mistaken equation is made, we find, in 8 or 9 of every 10 articles appearing in the leading journals of science, economics to medicine. The history of this "standard error" of science involves varied characters and plot twists, but especially R. A. Fisher's canonical translation of "Student's" t. William S. Gosset aka "Student," who was for most of his life Head Experimental Brewer at Guinness, took an economic approach to the logic of uncertainty. Against Gosset's wishes his friend Fisher erased the consciously economic element, Gosset's "real error." We want to bring it back. For the past eighty-five years it appears that some of the sciences have made a mistake, by basing decisions on statistical "significance." Though it looks at first like a matter of minor statistical detail, it is not. Statistics, magnitudes, coefficients are essential scientific tools. No one can credibly doubt that. And mathematical statistics is a glorious social and practical and aesthetic achievement. No one can credibly doubt that, either. Human understanding of chance and uncertainty would be much reduced were it not for Bayes's Theorem, gamma functions, the bell curve, and the rest. From the study of ancient parlor games to the rise of modern space science, mathematical statistics has shown its power. But one part of mathematical statistics has gone terribly wrong, though mostly unnoticed. The part we are worrying about here seems to have all the quantitative solidity and mathematical shine of the rest. But it also seems—unless we and some other observers of mathematical statistics such as Edgeworth, Gosset, Egon Pearson, Jeffreys, Borel, Neyman, Wald, Wolfowitz, Yule, Deming, Yates, L. J. Savage, de Finetti, Good, Lindley, Feynman, Lehmann, DeGroot, Bernardo, Chernoff, Raiffa, Arrow, Blackwell, Friedman, Mosteller, Kruskal, Mandelbrot, Wallis, Roberts, Granger, Leamer, Press, Moore, Berger, Gigerenzer, Freedman, Rothman, Zellner and a small town of others working in and around the American Statistical Association (see "Works Cited" in Ziliak and McCloskey 2008a, pp. 265-287) are quite mistaken—that reducing the scientific and commercial problems of testing, estimation and interpretation to one of "statistical significance," as some sciences have done for a hundred years, has been an exceptionally bad idea. Statistical significance is, we argue, a diversion from the proper objects of scientific study. Significance, reduced to its narrow and statistical meaning only—as in "low" observed "standard error" or "p < .05"—has little to do with a defensible notion of scientific inference, error analysis, or rational decision making. And yet in daily use it produces unchecked a large net loss for science and society. Its arbitrary, mechanical illogic, though currently sanctioned by science and its bureaucracies of reproduction, is causing a loss of jobs, justice, profit, and even life. Statistical significance at the 5% or other arbitrary level is neither necessary nor sufficient for proving discovery of a scientific or commercially relevant result. How the
Subfields: Artificial Intelligence, Economics and Econometrics, Statistics and Probability
View Paper: https://www.semanticscholar.org/paper/cebb4580a00a291fcf7d30f928c0e90e7689a908
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Statistics and Probability
Citations in the corpus
- The use of Bayes and causal modelling in decision making, uncertainty and risk (2011, background): “However, in addition to serious concerns about the use of p-values generally (as described comprehensively in [13]), there is an inevitable temptation arising from such results to infer causal links such as, in this case, higher temperatures cause more fatalities.”
- Fact and Fiction in EU-Governmental Economic Data (2011, background): “For this and other problems of statistical significance, see Krämer (2010). 4.”