A minimum description length framework for unsupervised learning

By Richard S. Zemel. 1994

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, neutral): “Many UL methods are designed to maximize information-theoretic objectives (e.g., Linsker, 1988; Barlow et al., 1989; MacKay and Miller, 1990; Plumbley, 1991; Schmidhuber, 1992b,c; Schraudolph and Sejnowski, 1993; Redlich, 1993; Zemel, 1993; Zemel and Hinton, 1994; Field, 1994; Hinton et al., 1995; D”
Stochastic Back-propagation and Variational Inference in Deep Latent Gaussian Models (2014, methodology): “We introduce an approximate posterior distribution q and applying Jensen’s inequality, following the variational principle (Zemel, 1993; Beal, 2003) to obtain: We use the variational free energy, deﬁned as F ( V ) = −L ( V ), as our objective function.”
Learning Dynamic Bayesian Networks (1997, methodology): “This static mixture model, without inclusion of the t ime index and the Markov dynamics, is a factorial parameterization of the standard mixture of Gaussians model that has interest in its own right [52, 20, 14].”
Reconciling simplicity and likelihood principles in perceptual organization. (1996, background): “These ideas have also been applied in the literatures on machine learning (eg, Quinlan & Rivest, 1989),neural networks (eg, Zemel, 1993),and even to problems quite closely connected to perceptual organization: automatic handwritten character recognition (Goa & Li, 1989)and computer vision approaches”
The Helmholtz Machine (1995, methodology): “A recognition model is used to infer a probability distribution over the underlying causes from the sensory input, and a separate generative model, which is also learned, is used to train the recognition model (Zemel, 1994; Hinton & Zemel, 1994; Zemel & Hinton, 1994).”
Feature Extraction Through LOCOCODE (1999, methodology): “This task is adapted from Dayan and Zemel (1995) (see also Földiák, 1990; Zemel, 1993; Saund, 1995) but is more difficult (compare Baumgartner, 1996).”
Helmholtz Machines and Wake-Sleep Learning (2000, methodology): “Congeners include probabilistic autoencoders (see Zemel, 1994); forward-inverse models (Kawato, Hayakama and Inui, 1993), maximum likelihood Independent Component Analysis (Bell and Sejnowski, 1995), sparse coding models (Olshausen and Field, 1996), hierarchical Gaussian models (Luettgen and…”
Generalization in Interactive Networks: The Benefits of Inhibitory Competition and Hebbian Learning (2001, methodology): “Specifically, the input vocabulary consists of all 45 combinations of 5 horizontal and 5 vertical bars in a 5 × 5 grid, and the output mapping is a localist identification of the two input bars (this is similar to the bars tasks used by Földiák, 1990; Saund, 1995; Zemel, 1993; Dayan & Zemel, 1995).”
Sparse coding with an overcomplete basis set: a strategy employed by V1? (1997, methodology): “Other methods for learning sparse codes have been described by Foldiak (1990) and Zemel (1993) .”
Varieties of Helmholtz Machine (1996, methodology): “A HM with just one hidden layer is exactly (a folded-over version of) the sort of autoencoder that Zemel (1994) and Hinton & Zemel (1994) treated.”
Competition and Multiple Cause Models (1995, methodology): “Another approach is to encourage sparsity in the activities of the hidden units, eg Földiák (1990) and Zemel (1993).”
Developing Population Codes by Minimizing Description Length (1993, methodology): “We can determine the instantiation parameters of this particular shape by computing the center of gravity of the blob activities, marked here by an \X".”
An Efficient Learning Procedure for Deep Boltzmann Machines (2012, background): “In variational learning (Zemel, 1993; Hinton & Zemel, 1994; Neal & Hinton, 1998; Jordan et al., 1999), the true posterior distribution over latent variables P ( h | v ; θ) for each training vector v is replaced by an approximate posterior Q ( h | v ; μ) , and the parameters are updated to maximize…”