The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression
By E.J. Candes, Pragya Sur. Annals of Statistics, 2018
152 citations recorded by the source corpus.
This paper rigorously establishes that the existence of the maximum likelihood estimate (MLE) in high-dimensional logistic regression models with Gaussian covariates undergoes a sharp `phase transition'. We introduce an explicit boundary curve $h_{\text{MLE}}$, parameterized by two scalars measuring the overall magnitude of the unknown sequence of regression coefficients, with the following property: in the limit of large sample sizes $n$ and number of features $p$ proportioned in such a way that $p/n \rightarrow \kappa$, we show that if the problem is sufficiently high dimensional in the sense that $\kappa > h_{\text{MLE}}$, then the MLE does not exist with probability one. Conversely, if $\kappa < h_{\text{MLE}}$, the MLE asymptotically exists with probability one.
Subfields: Artificial Intelligence, Statistics and Probability
View Paper: https://www.semanticscholar.org/paper/a8a55766b61e2a122c0153cf1fc3a0cdbacbc2bd
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Lenka Zdeborov'a
- Florent Krzakala
- E.J. Candes
- Robert Tibshirani
- Pragya Sur
- Alnur Ali
- Sebastian Goldt
- M. M'ezard
In Statistics and Probability
- E.J. Candes
- Pragya Sur
- Robert Tibshirani
- Alnur Ali
- Sebastian Goldt
- Lenka Zdeborov'a
- Florent Krzakala
- Martin J. Wainwright
Citations in the corpus
- A modern maximum-likelihood theory for high-dimensional logistic regression (2018, background): “It turns out that a companion paper (39) precisely characterizes the region in which the MLE exists.”
- Capturing the learning curves of generic features maps for realistic data sets with a teacher-student model (2021, background): “In the wake of understanding the limitations of classical statistical learning approaches [8, 9, 10] this direction is witnessing a renewal of interest [11, 12, 10, 13, 10, 14, 15].”
- The Generalized Lasso Problem and Uniqueness (2018, background): “2 Related work Several authors have examined uniqueness of solutions in statistical optimization problems en route to proving risk or recovery properties of these solutions; see Donoho (2006); Dossal (2012) for examples of this in the noiseless lasso problem (and the analogous noiseless `0 penalized”