Modeling pixel means and covariances using factorized third-order boltzmann machines

By Marc'Aurelio Ranzato, Geoffrey E. Hinton. 2010

261 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Biophysics, Cognitive Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biophysics

In Cognitive Neuroscience

Citations in the corpus

Disentangling Factors of Variation via Generative Entangling (2012, methodology): “…together from purely unsupervised data, and thereby extract directions of variance over which the pooling features become invariant (Kohonen et al., 1979; Ko-honen, 1996; Hyv¨arinen and Hoyer, 2000; Le et al., 2010; Kavukcuoglu et al., 2009; Ranzato and Hinton, 2010; Courville et al., 2011b).”
Deep Learning: Methods and Applications (2014, background): “The standard DBN has been extended to the factored higher-order Boltzmann machine in its bottom layer, with strong results obtained for phone recognition [64] and for computer vision [296].”
A Kronecker-factored approximate Fisher matrix for convolution layers (2016, methodology): “In the tradition of fast food puns (Ranzato & Hinton, 2010; Yang et al., 2014), we call our method Kronecker Factors for Convolution (KFC).”
Learning Where to Attend with Deep Architectures for Image Tracking (2011, methodology): “For the ﬁrst layers, we use (factored)-RBMs (Hinton & Salakhutdinov, 2006; Ranzato & Hinton, 2010; Welling et al., 2005; Swersky et al., 2011), but autoencoders (Vincent et al., 2008), sparse coding (Olshausen & Field, 1996; Kavukcuoglu et al., 2009), two-layer ICA (K¨oster & Hyv¨arinen, 2007) and…”
N4-Fields: Neural Network Nearest Neighbor Fields for Image Transforms (2014, methodology): “Kivinen et al. [10] proposed using unsupervised features extraction via deep belief net extension of mcRBM [11] followed by supervised NN training for boundary prediction in natural images.”
Learning to Relate Images (2011, background): “Recently, [32] showed how multiplicative interactions between a class-label and a feature vector can be viewed as an invariant classiﬁer, where each class is represented by a manifold of allowable transformations.”
The Deep Tensor Neural Network With Applications to Large Vocabulary Speech Recognition (2013, methodology): “The tensor-based restricted Boltzmann machine (RBM) was proposed to model three-way interactions among pixels in [16] and to model pixel means and covariances in [17].”
Visual Boundary Prediction: A Deep Neural Prediction Network and Quality Dissection (2014, methodology): “The mcRBM-model [Ranzato and Hinton, 2010] is a generative model for images.”
Object recognition with hierarchical kernel descriptors (2011, background): “Very recently, factorized third-order restricted Boltzmann machine [25] (mcRBM) have been introduced to capture high order dependencies of images.”
Learning hierarchical representations for face verification with convolutional deep belief networks (2012, methodology): “These methods have been successfully applied to visual recognition tasks [29, 41, 16, 39, 28, 12].”
Learning Deep Energy Models (2011, methodology): “In contrast to DBNs, DBMs have undirected connections between all layers of the network (Fig.”
Texture Modeling with Convolutional Spike-and-Slab RBMs and Deep Extensions (2012, methodology): “For unlike models such as the mcRBM (Ranzato and Hinton, 2010) and mPoT (Ranzato et al.”
A Spike and Slab Restricted Boltzmann Machine (2011, background): “Ranzato and Hinton (2010) have argued that the failure of the GRBM to adequately capture the statistical structure apparent in natural images stems from the exclusive use of the model capacity to capture the conditional mean at the expense of the conditional covariance.”
Phone Recognition with the Mean-Covariance Restricted Boltzmann Machine (2010, methodology): “Inspired by recent successes in modeling natural images, the primary contribution of this work is to bring the mean-covariance restricted Boltzmann machine (mcRBM) of [6] to bear on the problem of extracting useful features for phone recognition and to incorporate these features into a deep architec”
A Winner-Take-All Method for Training Sparse Convolutional Autoencoders (2014, unknown): “3% Mean-covariance RBM (3 layers) [22] 71.”