Information Theory and an Extension of the Maximum Likelihood Principle

By Hirotogu Akaike, Hirotugu Akaike. 1973

Subfields: Statistics and Probability, Information Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

In Information Theory

Citations in the corpus

Deep learning in neural networks: An overview (2014, neutral): “…sets (Craven & Wahba, 1979; Eubank, 1988; Golub, Heath, & Wahba, 1979; Hastie & Tibshirani, 1990; Mosteller & Tukey, 1968; Stone, 1974), Akaike’s information criterion and final prediction error (Akaike, 1970, 1973, 1974), or generalized prediction error (Moody, 1992; Moody & Utans, 1994).”
Learning Sparse Neural Networks through L0 Regularization (2017, methodology): “The Akaike Information Criterion (AIC) (Akaike, 1998) and the Bayesian Information Criterion (BIC) (Schwarz et al.”
Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems (2019, background): “The mutual information between ( x , y ) and z satisﬁes the information theoretic identity where H ( x , y ) , H ( z ) are the marginal entropies and H ( x , y | z ) , H ( z | x , y ) are the conditional entropies [56].”
Fused Lasso Additive Model (2014, methodology): “As an alternative, an approach for tuning parameter selection based on Akaike’s information criterion (AIC; Akaike 1973) or Bayesian information criterion (BIC; Schwarz 1978) could be developed.”
Adversarial Uncertainty Quantification in Physics-Informed Neural Networks (2018, background): “The mutual information between ( x , t, u ) and z satisﬁes the information theoretic identity where h ( x , t, u ), h ( z ) are the marginal entropies and h ( x , t, u | z ), h ( z | x , t, u ) are the conditional entropies [36].”
Nearly unbiased variable selection under minimax concave penalty (2010, methodology): “For example, one may impose a proper penalty on the number of selected variables based on the AIC [Akaike (1973)], Cp [Mallows (1973)], BIC [Schwarz (1978)], RIC [Foster and George (1994)] or a data driven method.”
Risk hull method and regularization by projections of ill-posed inverse problems (2006, unknown): “The Annals of Statistics 2006, Vol. 34, No. 4, 1653–1677 DOI: 10.1214/009053606000000542 © Institute of Mathematical Statistics, 2006”
Mixtures of Experts Models (2018, methodology): “Interestingly, in the model with the highest AICM value, there are covariates in the cluster membership probabilities as well as in the link probabilities.”
An Introduction to Model Selection. (2000, background): “This is a somewhat disguised, but strictly equivalent , form of the well-known Akaike information criterion (Akaike, 1973).”
Estimating abundance of killer whales in the nearshore waters of the Gulf of Alaska and Aleutian Islands using line-transect sampling (2007, methodology): “ranked according to the Akaike Information Criterion, which provides a measure of model Wt with a penalty term for the number of parameters in the model (Akaike 1973).”
Foundations of Linear and Generalized Linear Models (2015, background): “Akaike (1973) showed that when M is reasonably close to the true model, the maximized log likelihood L(βM) for M is a biased estimator of E{E log[pM(y; βM)]}, and for large sample sizes the bias is reduced by subtracting the number of parameters in M.”
Misspecifying the Shape of a Random Effects Distribution: Why Getting It Wrong May Not Matter (2011, methodology): “Neuhaus, Hauck and Kalbfleisch (1992), Neuhaus, Kalbfleisch and Hauck (1994) and Heagerty and Kurland (2001) developed analytic results to assess the impact of misspecification of the random effects distribution using the theory of inference under misspecified models (Huber, 1967; Akaike, 1973; Whit”
Covert and overt expressions of self‐criticism and perfectionism and their relation to depression (2004, methodology): “In addition to 2 and the ratio of 2 to degrees of freedom, we report four other indices: the goodness of fit index (GFI) of Jöreskog and Sörbom (1984), Bentler’s (1990) Comparative Fit Index (CFI), the root mean square error of approximation (RMSEA; Browne & Cudeck, 1993), and the Akaike Informati”