The epic story of maximum likelihood

By Stephen M. Stigler. 2007

141 citations recorded by the source corpus.

At a superficial level, the idea of maximum likelihood must be prehistoric: early hunters and gatherers may not have used the words "method of maximum likelihood" to describe their choice of where and how to hunt and gather, but it is hard to believe they would have been surprised if their method had been described in those terms. It seems a simple, even unassail- able idea: Who would rise to argue in favor of a method of minimum likeli- hood, or even mediocre likelihood? And yet the mathematical history of the topic shows this "simple idea" is really anything but simple. Joseph Louis Lagrange, Daniel Bernoulli, Leonard Euler, Pierre Simon Laplace and Carl Friedrich Gauss are only some of those who explored the topic, not always in ways we would sanction today. In this article, that history is reviewed from back well before Fisher to the time of Lucien Le Cam's dissertation. In the process Fisher's unpublished 1930 characterization of conditions for the con- sistency and efficiency of maximum likelihood estimates is presented, and the mathematical basis of his three proofs discussed. In particular, Fisher's derivation of the information inequality is seen to be derived from his work on the analysis of variance, and his later approach via estimating functions was derived from Euler's Relation for homogeneous functions. The reaction to Fisher's work is reviewed, and some lessons drawn.

Subfields: Applied Mathematics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/884c32cdcc1e4f31e6011c9208e73d39e2a3d402

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Mathematics

In Statistics and Probability

Citations in the corpus

Monte Carlo Gradient Estimation in Machine Learning (2019, methodology): “The score function is important since, amongst other uses, it is the key quantity in maximum likelihood estimation (Stigler, 2007).”
Estimation Methods for Nonhomogeneous Regression Models: Minimum Continuous Ranked Probability Score versus Maximum Likelihood (2018, methodology): “In the field of statistics, regression coefficients and distribution parameters have traditionally mostly been estimated with maximum likelihood estimation (Aldrich 1997; Stigler 2007).”
Contrastive Pessimistic Likelihood Estimation for Semi-Supervised Classification (2015, methodology): “ML estimation is employed in ﬁelds as diverse as genealogy, imaging, genetics, astrophysics, physiology, and quantum communication, as is illustrated by many recent research works such as [5–17].”
Minimax Estimation of Functionals of Discrete Distributions (2014, unknown): “pective of mathematical statistics, however, maximum likelihood is by no means sacrosanct. As early as in 1930, in his letters to Fisher, Hotelling raised the possibility of the MLE performing poorly [150]. Subsequently, various examples showing that the performance of the MLE can be signiﬁcantly im”
Mere Renovation is Too Little Too Late: We Need to Rethink our Undergraduate Curriculum from the Ground Up (2015, methodology): “(In his mathematical vision, Fisher had precursors (Laplace’s use of Bayesian methods, Galton’s and Pearson’s parametric families), but those were hardly determinative of mathematics as a unifying principle (see Stigler 2005, 2007).”