Classification by Minimum-Message-Length Inference

By Chris S. Wallace. 1991

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

Citations in the corpus

Evaluating Lossy Compression Rates of Deep Generative Models (2020, methodology): “The basic scheme was proposed by Wallace (1990); Hinton & Van Camp (1993), and later implemented by Frey & Hinton (1996).”
Bayesian Compression for Deep Learning (2017, background): “It is well understood that variational inference can be reinterpreted from an MDL point of view [56, 72, 28, 30, 19].”
iVPF: Numerical Invertible Volume Preserving Flow for Efficient Lossless Compression (2021, methodology): “VAEs leverage the latent variables for expressive explicit mixture models and employ bits-back techniques [38, 12] to encode the hierarchical distribution structure.”
Bit-Swap: Recursive Bits-Back Coding for Lossless Compression with Hierarchical Latent Variables (2019, methodology): “Wallace (1990) and Hinton & Van Camp (1993) show in a thought experiment, called the bits-back argument, it is possible to instead transmit − log qθ(z|x) fewer bits in a certain sense, thereby yielding a better net codelength equal to the negative ELBO −L(θ) of the latent variable model.”
Latent Variable Models (2019, methodology): “Bits back coding (Wallace, 1990; Hinton & van Camp, 1993) is a method for performing lossless compression using a latent variable model.”
HiLLoC: Lossless Image Compression with Hierarchical Latent Variable Models (2019, methodology): “Bits back coding (Wallace, 1990; Hinton & van Camp, 1993) is a method for performing lossless compression using a latent variable model.”
Practical Lossless Compression with Latent Variables using Bits Back Coding (2019, methodology): “However there is a method, known as ‘bits back coding’ (Wallace, 1990; Hinton and Camp, 1993), first introduced as a thought experiment, but later implemented in Frey and Hinton (1996) and Frey (1997), which can be used to extend those algorithms to cope with latent variables.”
MML clustering of multi-state, Poisson, von Mises circular and Gaussian distributions (2000, background): “Thereasonforthisisthat(Wallace1986 (Section3),Wallace1990,p.77)if p ( j ; x ) ; j D 1 ; : : : ; J , is the probability of component j generating datum x , then the total assignment of x to its best component results in a message length of ¡ log(max j p ( j ; x )) to encode x whereas, letting P ( x…”
Improving Inference for Neural Image Compression (2020, methodology): “A similar marginalization gap has been addressed in lossless compression by bitsback coding [Wallace, 1990, Hinton and Van Camp, 1993] and made more practical by the BB-ANS algorithm [Townsend et al., 2019a].”
Lossy Compression for Lossless Prediction (2021, background): “Bits-back coding (Wallace, 1990) can efficiently reach the desired bit-rate only because it is in the lossless setting.”