Information and Complexity in Statistical Modeling

By Jorma Rissanen. 2006

384 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Statistical and Nonlinear Physics

Citations in the corpus

Weight Agnostic Neural Networks (2019, methodology): “The Minimal Description Length (MDL) [34, 91, 92] is a formalization of Occam’s razor, in which a good model is one that is best at compressing its data, including the cost of describing of the model itself.”
Improved contact prediction in proteins: using pseudolikelihoods to infer Potts models. (2012, background): “Oja, Independent Component Analysis, Adaptive and Learning Systems for Signal Processing, Communications, and Control (John Wiley & Sons, 2001) [25] J.”
The Description Length of Deep Learning models (2018, methodology): “• We introduce new practical ways to compute tight compression bounds in deep learning models, based on the MDL toolbox (Grünwald, 2007; Rissanen, 2007).”
Unsupervised learning of compositional sparse code for natural image representation (2013, background): “One is from the minimum description length (MDL) [30] perspective, where γ can be interpreted as the cost of coding the perturbations (Δxm,i,Δαm,i) of Bxi,s,αi in the encoding of Im.”
Scalable clustering of news search results (2011, methodology): “This code length corresponds to a special encoding scheme inspired by the Minimum Description Length principle[16, 8].”
Hierarchical block structures and high-resolution model selection in large networks (2013, background): “Its size and partition are chosen exactly as in the resize move above.”
Summarizing categorical data by clustering attributes (2011, methodology): “If for one reason or another it would be required to explicitly include the cost of these values into the total description length, one could do so by using a Universal Code for integers (Rissanen, 2007).”
AIC model selection and multimodel inference in behavioral ecology: some background, observations, and comparisons (2011, background): “In addition Rissanen (2007) provides alternative approaches also based on information theory, but from a coding theory standpoint, whereas Konishi and Kitagawa (2007) continue to emphasize the link between information theory and statistical modeling.”
Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes (2017, background): “This intuitive explanation is called Occam’s razor, and No Free Lunch theorem (Shalev-Shwartz & Ben-David, 2014) and also related to the minimum description length (MDL) theory (Rissanen, 2007; Myung, 2000).”
Information, Divergence and Risk for Binary Experiments (2009, neutral): “…1994), information-theoretic (Jenssen, 2005a; Harremoës, 1993), game-theoretic (Vovk et al., 2005; Grünwald and Dawid, 2004), MDL (Grünwald, 2007; Rissanen, 2007), regularised distance minimisation (Borwein and Lewis, 1991; Altun and Smola, 2006; Broniatowski, 2004), and more narrowly…”
Model Selection in Linear Mixed Models (2013, background): “The minimum description length approach (MDL) developed by Rissanen in the 1980s (see Rissanen (2007)) chooses the model that achieves maximum data compression by minimizing the code length of the data and the model.”
An analysis of global warming in the Alpine region based on nonlinear nonstationary time series models (2012, positive): “We assume, as in Rissanen (2007) that there is no absolutely correct model, but only models with a better or worse fit.”