Minimum Description Length Principle.

By Jorma Rissanen. 2010

1,403 citations recorded by the source corpus.

Subfields: Computational Theory and Mathematics, Physical and Theoretical Chemistry, Scaling & Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Physical and Theoretical Chemistry

In Scaling & Learning Theory

Citations in the corpus

Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameters (2018, methodology): “The bits-back argument (Hinton & Van Camp, 1993) establishes a connection between the Bayesian variational framework and the Minimum Description Length (MDL) principle (Gr¨unwald, 2007).”
PAC-Bayesian Theory Meets Bayesian Inference (2016, positive): “The analysis of Grünwald [16] relies both on the Minimum Description Length principle [19] and PAC-Bayesian theory.”
Weight Agnostic Neural Networks (2019, methodology): “The Minimal Description Length (MDL) [34, 91, 92] is a formalization of Occam’s razor, in which a good model is one that is best at compressing its data, including the cost of describing of the model itself.”
Agnostic Federated Learning (2019, neutral): “Similar results are known for universal compression, where the goal is to compress a sequence of random variables without knowledge of the generating distribution (Gr¨unwald, 2007).”
Distinguishing Cause from Effect Using Observational Data: Methods and Benchmarks (2014, methodology): “…(empirical) Bayesian marginal likelihoods can be interpreted as measuring likelihood in combination with a complexity penalty, Minimum Message Length (MML) techniques can be used to construct scores that incorporate a trade-off between model fit (likelihood) and model complexity (Grünwald, 2007).”
Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies (2018, methodology): “These approaches can broadly be seen as efficient implementations of the Minimum Description Length (MDL) principle for unsupervised learning [44, 18].”
Causal Inference Using the Algorithmic Markov Condition (2008, background): “In particular, minimum description length (MDL) approaches [21] appear promising from the theoretical point of view due to their close relation to Kolmogorov complexity.”
Bayesian Compression for Deep Learning (2017, methodology): “A fundamental theorem in information theory is the minimum description length (MDL) principle [20].”
The Description Length of Deep Learning models (2018, methodology): “…under strong assumptions ( d -dimensional identiﬁable model, data coming from the model, suitable Bayesian prior, and technical assumptions ensuring the effective dimension of the trained model is not lower than d ): in that case, these three methods yield a codelength L ( y (Grünwald, 2007).”
Further Analysis of Outlier Detection with Deep Generative Models (2020, positive): “The atypicality test considers a test sequence to be OOD when there exists an alternative model leading to a smaller description length [34].”
Keeping it Short and Simple: Summarising Complex Event Sequences with Multivariate Patterns (2015, methodology): “To ﬁnd such sets, we employ the Minimum Description Length (MDL) principle [8].”
User-friendly introduction to PAC-Bayes bounds (2021, background): “We refer the reader to [24, 87] for more recent presentations of this very fruitful approach.”
A Brief Introduction to Machine Learning for Engineers (2017, background): “The reader is warned that the treatment here is rather superficial, and that a more formal definition of the MDL criterion would would require a more sophisticated discussion, which can be found in [41].”
Estimating means of bounded random variables by betting (2020, background): “In particular, Shafer espouses complementary beneﬁts of such approaches, ranging from improved scientiﬁc communication, ties to historical advances in probability, and reproducibility via continued experimentation (also see Grünwald et al. [2019]), but our focus here has been on developing a new sta”
Comprehensible classification models: a position paper (2014, background): “works on multi-objective optimization concepts such as Pareto dominance [25], [38] and lexicographic optimization [25], [42]; the minimum description length principle [30]; and Occam’s razor [17], [74].”